You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2017/02/12 04:23:39 UTC
[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build #
1126 - Still Failing!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1126/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseSerialGC
1 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestTolerantUpdateProcessorRandomCloud
Error Message:
2 threads leaked from SUITE scope at org.apache.solr.cloud.TestTolerantUpdateProcessorRandomCloud: 1) Thread[id=30574, name=OverseerHdfsCoreFailoverThread-97443614124081172-127.0.0.1:59066_solr-n_0000000004, state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.] at java.lang.Thread.sleep(Native Method) at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:139) at java.lang.Thread.run(Thread.java:745) 2) Thread[id=30572, name=OverseerStateUpdate-97443614124081172-127.0.0.1:59066_solr-n_0000000004, state=TIMED_WAITING, group=Overseer state updater.] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:164) at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:138) at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:179) at java.lang.Thread.run(Thread.java:745)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.TestTolerantUpdateProcessorRandomCloud:
1) Thread[id=30574, name=OverseerHdfsCoreFailoverThread-97443614124081172-127.0.0.1:59066_solr-n_0000000004, state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.]
at java.lang.Thread.sleep(Native Method)
at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:139)
at java.lang.Thread.run(Thread.java:745)
2) Thread[id=30572, name=OverseerStateUpdate-97443614124081172-127.0.0.1:59066_solr-n_0000000004, state=TIMED_WAITING, group=Overseer state updater.]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:164)
at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:138)
at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:179)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([E880CF9BE662A713]:0)
Build Log:
[...truncated 12534 lines...]
[junit4] Suite: org.apache.solr.cloud.TestTolerantUpdateProcessorRandomCloud
[junit4] 2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_E880CF9BE662A713-001/init-core-data-001
[junit4] 2> 2786233 INFO (SUITE-TestTolerantUpdateProcessorRandomCloud-seed#[E880CF9BE662A713]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields
[junit4] 2> 2786235 INFO (SUITE-TestTolerantUpdateProcessorRandomCloud-seed#[E880CF9BE662A713]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-9182 - causes OOM)
[junit4] 2> 2786235 INFO (SUITE-TestTolerantUpdateProcessorRandomCloud-seed#[E880CF9BE662A713]-worker) [ ] o.a.s.c.TestTolerantUpdateProcessorRandomCloud Configuring cluster: servers=7, shards=3, repfactor=2
[junit4] 2> 2786236 INFO (SUITE-TestTolerantUpdateProcessorRandomCloud-seed#[E880CF9BE662A713]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 7 servers in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_E880CF9BE662A713-001/tempDir-001
[junit4] 2> 2786236 INFO (SUITE-TestTolerantUpdateProcessorRandomCloud-seed#[E880CF9BE662A713]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 2786236 INFO (Thread-7277) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2786237 INFO (Thread-7277) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 2786336 INFO (SUITE-TestTolerantUpdateProcessorRandomCloud-seed#[E880CF9BE662A713]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:54088
[junit4] 2> 2786345 INFO (jetty-launcher-5461-thread-1) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 2786346 INFO (jetty-launcher-5461-thread-4) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 2786346 INFO (jetty-launcher-5461-thread-6) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 2786346 INFO (jetty-launcher-5461-thread-2) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 2786346 INFO (jetty-launcher-5461-thread-5) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 2786347 INFO (jetty-launcher-5461-thread-3) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 2786347 INFO (jetty-launcher-5461-thread-7) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 2786352 INFO (jetty-launcher-5461-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@584fcbbd{/solr,null,AVAILABLE}
[junit4] 2> 2786353 INFO (jetty-launcher-5461-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@709f0bc6{/solr,null,AVAILABLE}
[junit4] 2> 2786353 INFO (jetty-launcher-5461-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5ea94616{/solr,null,AVAILABLE}
[junit4] 2> 2786354 INFO (jetty-launcher-5461-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@332a6509{HTTP/1.1,[http/1.1]}{127.0.0.1:55101}
[junit4] 2> 2786354 INFO (jetty-launcher-5461-thread-1) [ ] o.e.j.s.Server Started @2789849ms
[junit4] 2> 2786354 INFO (jetty-launcher-5461-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@40e27e84{HTTP/1.1,[http/1.1]}{127.0.0.1:61589}
[junit4] 2> 2786354 INFO (jetty-launcher-5461-thread-4) [ ] o.e.j.s.Server Started @2789849ms
[junit4] 2> 2786354 INFO (jetty-launcher-5461-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=61589}
[junit4] 2> 2786354 INFO (jetty-launcher-5461-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=55101}
[junit4] 2> 2786354 ERROR (jetty-launcher-5461-thread-4) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2786354 ERROR (jetty-launcher-5461-thread-1) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2786355 INFO (jetty-launcher-5461-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 2786355 INFO (jetty-launcher-5461-thread-5) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@19071077{/solr,null,AVAILABLE}
[junit4] 2> 2786355 INFO (jetty-launcher-5461-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2786355 INFO (jetty-launcher-5461-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2786355 INFO (jetty-launcher-5461-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-12T03:52:33.524Z
[junit4] 2> 2786355 INFO (jetty-launcher-5461-thread-6) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4deaa889{/solr,null,AVAILABLE}
[junit4] 2> 2786355 INFO (jetty-launcher-5461-thread-5) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2f600e9{HTTP/1.1,[http/1.1]}{127.0.0.1:64430}
[junit4] 2> 2786355 INFO (jetty-launcher-5461-thread-5) [ ] o.e.j.s.Server Started @2789850ms
[junit4] 2> 2786355 INFO (jetty-launcher-5461-thread-5) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=64430}
[junit4] 2> 2786355 INFO (jetty-launcher-5461-thread-6) [ ] o.e.j.s.AbstractConnector Started ServerConnector@f1edc30{HTTP/1.1,[http/1.1]}{127.0.0.1:59066}
[junit4] 2> 2786355 ERROR (jetty-launcher-5461-thread-5) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2786355 INFO (jetty-launcher-5461-thread-6) [ ] o.e.j.s.Server Started @2789850ms
[junit4] 2> 2786355 INFO (jetty-launcher-5461-thread-6) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=59066}
[junit4] 2> 2786355 INFO (jetty-launcher-5461-thread-5) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 2786356 INFO (jetty-launcher-5461-thread-5) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2786356 ERROR (jetty-launcher-5461-thread-6) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2786356 INFO (jetty-launcher-5461-thread-5) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2786356 INFO (jetty-launcher-5461-thread-5) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-12T03:52:33.525Z
[junit4] 2> 2786356 INFO (jetty-launcher-5461-thread-6) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 2786356 INFO (jetty-launcher-5461-thread-6) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2786356 INFO (jetty-launcher-5461-thread-6) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2786356 INFO (jetty-launcher-5461-thread-6) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-12T03:52:33.525Z
[junit4] 2> 2786356 INFO (jetty-launcher-5461-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@675a81cc{HTTP/1.1,[http/1.1]}{127.0.0.1:39480}
[junit4] 2> 2786356 INFO (jetty-launcher-5461-thread-2) [ ] o.e.j.s.Server Started @2789851ms
[junit4] 2> 2786356 INFO (jetty-launcher-5461-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=39480}
[junit4] 2> 2786356 ERROR (jetty-launcher-5461-thread-2) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2786357 INFO (jetty-launcher-5461-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 2786357 INFO (jetty-launcher-5461-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2786357 INFO (jetty-launcher-5461-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2786357 INFO (jetty-launcher-5461-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-12T03:52:33.526Z
[junit4] 2> 2786358 INFO (jetty-launcher-5461-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@485331aa{/solr,null,AVAILABLE}
[junit4] 2> 2786358 INFO (jetty-launcher-5461-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@147cd8ee{HTTP/1.1,[http/1.1]}{127.0.0.1:41793}
[junit4] 2> 2786358 INFO (jetty-launcher-5461-thread-3) [ ] o.e.j.s.Server Started @2789853ms
[junit4] 2> 2786358 INFO (jetty-launcher-5461-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=41793}
[junit4] 2> 2786359 ERROR (jetty-launcher-5461-thread-3) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2786359 INFO (jetty-launcher-5461-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 2786359 INFO (jetty-launcher-5461-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2786359 INFO (jetty-launcher-5461-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2786359 INFO (jetty-launcher-5461-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-12T03:52:33.528Z
[junit4] 2> 2786355 INFO (jetty-launcher-5461-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 2786360 INFO (jetty-launcher-5461-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2786360 INFO (jetty-launcher-5461-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2786360 INFO (jetty-launcher-5461-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-12T03:52:33.529Z
[junit4] 2> 2786363 INFO (jetty-launcher-5461-thread-7) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@62e9f5ff{/solr,null,AVAILABLE}
[junit4] 2> 2786364 INFO (jetty-launcher-5461-thread-7) [ ] o.e.j.s.AbstractConnector Started ServerConnector@31cf3a4e{HTTP/1.1,[http/1.1]}{127.0.0.1:33012}
[junit4] 2> 2786364 INFO (jetty-launcher-5461-thread-7) [ ] o.e.j.s.Server Started @2789859ms
[junit4] 2> 2786364 INFO (jetty-launcher-5461-thread-7) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33012}
[junit4] 2> 2786364 ERROR (jetty-launcher-5461-thread-7) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2786364 INFO (jetty-launcher-5461-thread-7) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 2786364 INFO (jetty-launcher-5461-thread-7) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2786364 INFO (jetty-launcher-5461-thread-7) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2786364 INFO (jetty-launcher-5461-thread-7) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-12T03:52:33.533Z
[junit4] 2> 2786365 INFO (jetty-launcher-5461-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2786369 INFO (jetty-launcher-5461-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2786369 INFO (jetty-launcher-5461-thread-6) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2786371 INFO (jetty-launcher-5461-thread-5) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2786372 INFO (jetty-launcher-5461-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2786380 INFO (jetty-launcher-5461-thread-7) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2786382 INFO (jetty-launcher-5461-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2786387 INFO (jetty-launcher-5461-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54088/solr
[junit4] 2> 2786387 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15a3073657f0007, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 2786390 INFO (jetty-launcher-5461-thread-5) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54088/solr
[junit4] 2> 2786390 INFO (jetty-launcher-5461-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54088/solr
[junit4] 2> 2786393 INFO (jetty-launcher-5461-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54088/solr
[junit4] 2> 2786394 INFO (jetty-launcher-5461-thread-6) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54088/solr
[junit4] 2> 2786395 INFO (jetty-launcher-5461-thread-7) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54088/solr
[junit4] 2> 2786401 INFO (jetty-launcher-5461-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54088/solr
[junit4] 2> 2786405 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15a3073657f000d, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 2786406 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15a3073657f000b, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 2786408 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15a3073657f0011, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 2786472 INFO (jetty-launcher-5461-thread-7) [n:127.0.0.1:33012_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:33012_solr
[junit4] 2> 2786473 INFO (jetty-launcher-5461-thread-3) [n:127.0.0.1:41793_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41793_solr
[junit4] 2> 2786473 INFO (jetty-launcher-5461-thread-1) [n:127.0.0.1:55101_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55101_solr
[junit4] 2> 2786474 INFO (jetty-launcher-5461-thread-7) [n:127.0.0.1:33012_solr ] o.a.s.c.Overseer Overseer (id=97443614124081171-127.0.0.1:33012_solr-n_0000000000) starting
[junit4] 2> 2786475 INFO (jetty-launcher-5461-thread-2) [n:127.0.0.1:39480_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39480_solr
[junit4] 2> 2786476 INFO (jetty-launcher-5461-thread-4) [n:127.0.0.1:61589_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:61589_solr
[junit4] 2> 2786479 INFO (zkCallback-5488-thread-1-processing-n:127.0.0.1:41793_solr) [n:127.0.0.1:41793_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2786480 INFO (zkCallback-5494-thread-1-processing-n:127.0.0.1:55101_solr) [n:127.0.0.1:55101_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2786492 INFO (zkCallback-5490-thread-1-processing-n:127.0.0.1:39480_solr) [n:127.0.0.1:39480_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2786492 INFO (zkCallback-5495-thread-1-processing-n:127.0.0.1:33012_solr) [n:127.0.0.1:33012_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2786492 INFO (zkCallback-5489-thread-1-processing-n:127.0.0.1:64430_solr) [n:127.0.0.1:64430_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2786492 INFO (zkCallback-5497-thread-1-processing-n:127.0.0.1:61589_solr) [n:127.0.0.1:61589_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2786494 INFO (zkCallback-5490-thread-1-processing-n:127.0.0.1:39480_solr) [n:127.0.0.1:39480_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (4)
[junit4] 2> 2786495 INFO (zkCallback-5494-thread-1-processing-n:127.0.0.1:55101_solr) [n:127.0.0.1:55101_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (4)
[junit4] 2> 2786500 INFO (zkCallback-5497-thread-2-processing-n:127.0.0.1:61589_solr) [n:127.0.0.1:61589_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (4)
[junit4] 2> 2786500 INFO (zkCallback-5488-thread-1-processing-n:127.0.0.1:41793_solr) [n:127.0.0.1:41793_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (4)
[junit4] 2> 2786500 INFO (zkCallback-5495-thread-2-processing-n:127.0.0.1:33012_solr) [n:127.0.0.1:33012_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2786500 INFO (zkCallback-5489-thread-1-processing-n:127.0.0.1:64430_solr) [n:127.0.0.1:64430_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (4)
[junit4] 2> 2786500 INFO (zkCallback-5495-thread-1-processing-n:127.0.0.1:33012_solr) [n:127.0.0.1:33012_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2786501 INFO (zkCallback-5496-thread-1-processing-n:127.0.0.1:59066_solr) [n:127.0.0.1:59066_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2786508 INFO (jetty-launcher-5461-thread-5) [n:127.0.0.1:64430_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:64430_solr
[junit4] 2> 2786510 INFO (zkCallback-5496-thread-2-processing-n:127.0.0.1:59066_solr) [n:127.0.0.1:59066_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (4)
[junit4] 2> 2786511 INFO (zkCallback-5488-thread-1-processing-n:127.0.0.1:41793_solr) [n:127.0.0.1:41793_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2786511 INFO (zkCallback-5490-thread-1-processing-n:127.0.0.1:39480_solr) [n:127.0.0.1:39480_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2786511 INFO (zkCallback-5494-thread-1-processing-n:127.0.0.1:55101_solr) [n:127.0.0.1:55101_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2786511 INFO (zkCallback-5489-thread-1-processing-n:127.0.0.1:64430_solr) [n:127.0.0.1:64430_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2786511 INFO (jetty-launcher-5461-thread-6) [n:127.0.0.1:59066_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59066_solr
[junit4] 2> 2786512 INFO (zkCallback-5497-thread-2-processing-n:127.0.0.1:61589_solr) [n:127.0.0.1:61589_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2786512 INFO (zkCallback-5495-thread-1-processing-n:127.0.0.1:33012_solr) [n:127.0.0.1:33012_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2786515 INFO (zkCallback-5496-thread-2-processing-n:127.0.0.1:59066_solr) [n:127.0.0.1:59066_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2786517 INFO (zkCallback-5490-thread-1-processing-n:127.0.0.1:39480_solr) [n:127.0.0.1:39480_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 2786517 INFO (zkCallback-5494-thread-1-processing-n:127.0.0.1:55101_solr) [n:127.0.0.1:55101_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 2786519 INFO (zkCallback-5497-thread-2-processing-n:127.0.0.1:61589_solr) [n:127.0.0.1:61589_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 2786519 INFO (zkCallback-5488-thread-1-processing-n:127.0.0.1:41793_solr) [n:127.0.0.1:41793_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 2786519 INFO (zkCallback-5495-thread-1-processing-n:127.0.0.1:33012_solr) [n:127.0.0.1:33012_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 2786519 INFO (zkCallback-5489-thread-1-processing-n:127.0.0.1:64430_solr) [n:127.0.0.1:64430_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 2786520 INFO (zkCallback-5496-thread-2-processing-n:127.0.0.1:59066_solr) [n:127.0.0.1:59066_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 2786536 INFO (jetty-launcher-5461-thread-7) [n:127.0.0.1:33012_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33012_solr
[junit4] 2> 2786550 INFO (zkCallback-5494-thread-1-processing-n:127.0.0.1:55101_solr) [n:127.0.0.1:55101_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (7)
[junit4] 2> 2786550 INFO (zkCallback-5488-thread-1-processing-n:127.0.0.1:41793_solr) [n:127.0.0.1:41793_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (7)
[junit4] 2> 2786556 INFO (zkCallback-5490-thread-1-processing-n:127.0.0.1:39480_solr) [n:127.0.0.1:39480_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (7)
[junit4] 2> 2786556 INFO (zkCallback-5495-thread-1-processing-n:127.0.0.1:33012_solr) [n:127.0.0.1:33012_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (7)
[junit4] 2> 2786556 INFO (zkCallback-5489-thread-1-processing-n:127.0.0.1:64430_solr) [n:127.0.0.1:64430_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (7)
[junit4] 2> 2786556 INFO (zkCallback-5497-thread-2-processing-n:127.0.0.1:61589_solr) [n:127.0.0.1:61589_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (7)
[junit4] 2> 2786569 INFO (zkCallback-5496-thread-2-processing-n:127.0.0.1:59066_solr) [n:127.0.0.1:59066_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (7)
[junit4] 2> 2786610 INFO (jetty-launcher-5461-thread-2) [n:127.0.0.1:39480_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_E880CF9BE662A713-001/tempDir-001/node2/.
[junit4] 2> 2786708 INFO (jetty-launcher-5461-thread-1) [n:127.0.0.1:55101_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_E880CF9BE662A713-001/tempDir-001/node1/.
[junit4] 2> 2786781 INFO (jetty-launcher-5461-thread-3) [n:127.0.0.1:41793_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_E880CF9BE662A713-001/tempDir-001/node6/.
[junit4] 2> 2786814 INFO (jetty-launcher-5461-thread-5) [n:127.0.0.1:64430_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_E880CF9BE662A713-001/tempDir-001/node5/.
[junit4] 2> 2786903 INFO (jetty-launcher-5461-thread-6) [n:127.0.0.1:59066_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_E880CF9BE662A713-001/tempDir-001/node4/.
[junit4] 2> 2786933 INFO (jetty-launcher-5461-thread-7) [n:127.0.0.1:33012_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_E880CF9BE662A713-001/tempDir-001/node7/.
[junit4] 2> 2786987 INFO (jetty-launcher-5461-thread-4) [n:127.0.0.1:61589_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_E880CF9BE662A713-001/tempDir-001/node3/.
[junit4] 2> 2790841 INFO (SUITE-TestTolerantUpdateProcessorRandomCloud-seed#[E880CF9BE662A713]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (7)
[junit4] 2> 2790842 INFO (SUITE-TestTolerantUpdateProcessorRandomCloud-seed#[E880CF9BE662A713]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:54088/solr ready
[junit4] 2> 2791565 INFO (qtp1629588653-30303) [n:127.0.0.1:39480_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&property.config=solrconfig-distrib-update-processor-chains.xml&property.schema=schema15.xml&collection.configName=solrCloudCollectionConfig&name=test_col&action=CREATE&numShards=3&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2791569 INFO (OverseerThreadFactory-11184-thread-1-processing-n:127.0.0.1:33012_solr) [n:127.0.0.1:33012_solr ] o.a.s.c.CreateCollectionCmd Create collection test_col
[junit4] 2> 2791682 INFO (qtp1571178598-30325) [n:127.0.0.1:33012_solr ] o.a.s.h.a.CoreAdminOperation core create command property.config=solrconfig-distrib-update-processor-chains.xml&property.schema=schema15.xml&qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&name=test_col_shard3_replica2&action=CREATE&numShards=3&collection=test_col&shard=shard3&wt=javabin&version=2
[junit4] 2> 2791682 INFO (qtp1629588653-30305) [n:127.0.0.1:39480_solr ] o.a.s.h.a.CoreAdminOperation core create command property.config=solrconfig-distrib-update-processor-chains.xml&property.schema=schema15.xml&qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&name=test_col_shard1_replica1&action=CREATE&numShards=3&collection=test_col&shard=shard1&wt=javabin&version=2
[junit4] 2> 2791684 INFO (qtp1676013452-30357) [n:127.0.0.1:41793_solr ] o.a.s.h.a.CoreAdminOperation core create command property.config=solrconfig-distrib-update-processor-chains.xml&property.schema=schema15.xml&qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&name=test_col_shard2_replica2&action=CREATE&numShards=3&collection=test_col&shard=shard2&wt=javabin&version=2
[junit4] 2> 2791685 INFO (qtp1438662574-30312) [n:127.0.0.1:61589_solr ] o.a.s.h.a.CoreAdminOperation core create command property.config=solrconfig-distrib-update-processor-chains.xml&property.schema=schema15.xml&qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&name=test_col_shard2_replica1&action=CREATE&numShards=3&collection=test_col&shard=shard2&wt=javabin&version=2
[junit4] 2> 2791691 INFO (qtp1964210261-30310) [n:127.0.0.1:59066_solr ] o.a.s.h.a.CoreAdminOperation core create command property.config=solrconfig-distrib-update-processor-chains.xml&property.schema=schema15.xml&qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&name=test_col_shard3_replica1&action=CREATE&numShards=3&collection=test_col&shard=shard3&wt=javabin&version=2
[junit4] 2> 2791692 INFO (qtp544137359-30328) [n:127.0.0.1:55101_solr ] o.a.s.h.a.CoreAdminOperation core create command property.config=solrconfig-distrib-update-processor-chains.xml&property.schema=schema15.xml&qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&name=test_col_shard1_replica2&action=CREATE&numShards=3&collection=test_col&shard=shard1&wt=javabin&version=2
[junit4] 2> 2791809 INFO (zkCallback-5494-thread-1-processing-n:127.0.0.1:55101_solr) [n:127.0.0.1:55101_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [7])
[junit4] 2> 2791809 INFO (zkCallback-5488-thread-1-processing-n:127.0.0.1:41793_solr) [n:127.0.0.1:41793_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [7])
[junit4] 2> 2791809 INFO (zkCallback-5496-thread-2-processing-n:127.0.0.1:59066_solr) [n:127.0.0.1:59066_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [7])
[junit4] 2> 2791809 INFO (zkCallback-5497-thread-2-processing-n:127.0.0.1:61589_solr) [n:127.0.0.1:61589_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [7])
[junit4] 2> 2791809 INFO (zkCallback-5490-thread-1-processing-n:127.0.0.1:39480_solr) [n:127.0.0.1:39480_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [7])
[junit4] 2> 2791810 INFO (zkCallback-5495-thread-3-processing-n:127.0.0.1:33012_solr) [n:127.0.0.1:33012_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [7])
[junit4] 2> 2792711 INFO (qtp1571178598-30325) [n:127.0.0.1:33012_solr c:test_col s:shard3 x:test_col_shard3_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2792718 INFO (qtp1571178598-30325) [n:127.0.0.1:33012_solr c:test_col s:shard3 x:test_col_shard3_replica2] o.a.s.s.ManagedIndexSchemaFactory The schema is configured as managed, but managed schema resource managed-schema not found - loading non-managed schema schema15.xml instead
[junit4] 2> 2792723 INFO (qtp1571178598-30325) [n:127.0.0.1:33012_solr c:test_col s:shard3 x:test_col_shard3_replica2] o.a.s.s.IndexSchema [test_col_shard3_replica2] Schema name=test
[junit4] 2> 2792730 INFO (qtp1629588653-30305) [n:127.0.0.1:39480_solr c:test_col s:shard1 x:test_col_shard1_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2792753 INFO (qtp1676013452-30357) [n:127.0.0.1:41793_solr c:test_col s:shard2 x:test_col_shard2_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2792755 INFO (qtp1964210261-30310) [n:127.0.0.1:59066_solr c:test_col s:shard3 x:test_col_shard3_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2792756 INFO (qtp1629588653-30305) [n:127.0.0.1:39480_solr c:test_col s:shard1 x:test_col_shard1_replica1] o.a.s.s.ManagedIndexSchemaFactory The schema is configured as managed, but managed schema resource managed-schema not found - loading non-managed schema schema15.xml instead
[junit4] 2> 2792763 INFO (qtp1964210261-30310) [n:127.0.0.1:59066_solr c:test_col s:shard3 x:test_col_shard3_replica1] o.a.s.s.ManagedIndexSchemaFactory The schema is configured as managed, but managed schema resource managed-schema not found - loading non-managed schema schema15.xml instead
[junit4] 2> 2792767 INFO (qtp1438662574-30312) [n:127.0.0.1:61589_solr c:test_col s:shard2 x:test_col_shard2_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2792790 INFO (qtp1676013452-30357) [n:127.0.0.1:41793_solr c:test_col s:shard2 x:test_col_shard2_replica2] o.a.s.s.ManagedIndexSchemaFactory The schema is configured as managed, but managed schema resource managed-schema not found - loading non-managed schema schema15.xml instead
[junit4] 2> 2792790 INFO (qtp1438662574-30312) [n:127.0.0.1:61589_solr c:test_col s:shard2 x:test_col_shard2_replica1] o.a.s.s.ManagedIndexSchemaFactory The schema is configured as managed, but managed schema resource managed-schema not found - loading non-managed schema schema15.xml instead
[junit4] 2> 2792795 INFO (qtp1676013452-30357) [n:127.0.0.1:41793_solr c:test_col s:shard2 x:test_col_shard2_replica2] o.a.s.s.IndexSchema [test_col_shard2_replica2] Schema name=test
[junit4] 2> 2792796 INFO (qtp1438662574-30312) [n:127.0.0.1:61589_solr c:test_col s:shard2 x:test_col_shard2_replica1] o.a.s.s.IndexSchema [test_col_shard2_replica1] Schema name=test
[junit4] 2> 2792820 INFO (qtp1964210261-30310) [n:127.0.0.1:59066_solr c:test_col s:shard3 x:test_col_shard3_replica1] o.a.s.s.IndexSchema [test_col_shard3_replica1] Schema name=test
[junit4] 2> 2792821 INFO (qtp1629588653-30305) [n:127.0.0.1:39480_solr c:test_col s:shard1 x:test_col_shard1_replica1] o.a.s.s.IndexSchema [test_col_shard1_replica1] Schema name=test
[junit4] 2> 2792826 INFO (qtp544137359-30328) [n:127.0.0.1:55101_solr c:test_col s:shard1 x:test_col_shard1_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2792843 INFO (qtp544137359-30328) [n:127.0.0.1:55101_solr c:test_col s:shard1 x:test_col_shard1_replica2] o.a.s.s.ManagedIndexSchemaFactory The schema is configured as managed, but managed schema resource managed-schema not found - loading non-managed schema schema15.xml instead
[junit4] 2> 2792937 INFO (qtp544137359-30328) [n:127.0.0.1:55101_solr c:test_col s:shard1 x:test_col_shard1_replica2] o.a.s.s.IndexSchema [test_col_shard1_replica2] Schema name=test
[junit4] 2> 2792992 WARN (qtp1438662574-30312) [n:127.0.0.1:61589_solr c:test_col s:shard2 x:test_col_shard2_replica1] o.a.s.s.IndexSchema [test_col_shard2_replica1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 2793031 WARN (qtp1964210261-30310) [n:127.0.0.1:59066_solr c:test_col s:shard3 x:test_col_shard3_replica1] o.a.s.s.IndexSchema [test_col_shard3_replica1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 2793066 WARN (qtp1629588653-30305) [n:127.0.0.1:39480_solr c:test_col s:shard1 x:test_col_shard1_replica1] o.a.s.s.IndexSchema [test_col_shard1_replica1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 2793072 WARN (qtp1571178598-30325) [n:127.0.0.1:33012_solr c:test_col s:shard3 x:test_col_shard3_replica2] o.a.s.s.IndexSchema [test_col_shard3_replica2] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 2793074 INFO (qtp1438662574-30312) [n:127.0.0.1:61589_solr c:test_col s:shard2 x:test_col_shard2_replica1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 2793090 INFO (qtp1629588653-30305) [n:127.0.0.1:39480_solr c:test_col s:shard1 x:test_col_shard1_replica1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 2793102 INFO (qtp1438662574-30312) [n:127.0.0.1:61589_solr c:test_col s:shard2 x:test_col_shard2_replica1] o.a.s.s.ManagedIndexSchema Created and persisted managed schema znode at /configs/solrCloudCollectionConfig/managed-schema
[junit4] 2> 2793102 INFO (qtp1571178598-30325) [n:127.0.0.1:33012_solr c:test_col s:shard3 x:test_col_shard3_replica2] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 2793107 INFO (qtp1438662574-30312) [n:127.0.0.1:61589_solr c:test_col s:shard2 x:test_col_shard2_replica1] o.a.s.s.ManagedIndexSchemaFactory After upgrading to managed schema in ZooKeeper, renamed the non-managed schema /configs/solrCloudCollectionConfig/schema15.xml to /configs/solrCloudCollectionConfig/schema15.xml.bak
[junit4] 2> 2793108 INFO (qtp1438662574-30312) [n:127.0.0.1:61589_solr c:test_col s:shard2 x:test_col_shard2_replica1] o.a.s.c.CoreContainer Creating SolrCore 'test_col_shard2_replica1' using configuration from collection test_col
[junit4] 2> 2793109 INFO (qtp1438662574-30312) [n:127.0.0.1:61589_solr c:test_col s:shard2 x:test_col_shard2_replica1] o.a.s.c.SolrCore [[test_col_shard2_replica1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_E880CF9BE662A713-001/tempDir-001/node3/test_col_shard2_replica1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_E880CF9BE662A713-001/tempDir-001/node3/./test_col_shard2_replica1/data/]
[junit4] 2> 2793109 INFO (qtp1438662574-30312) [n:127.0.0.1:61589_solr c:test_col s:shard2 x:test_col_shard2_replica1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3774df79
[junit4] 2> 2793110 INFO (qtp1438662574-30312) [n:127.0.0.1:61589_solr c:test_col s:shard2 x:test_col_shard2_replica1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=25, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=9.767578125, floorSegmentMB=0.3857421875, forceMergeDeletesPctAllowed=11.065629076629586, segmentsPerTier=30.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 2793114 INFO (qtp1629588653-30305) [n:127.0.0.1:39480_solr c:test_col s:shard1 x:test_col_shard1_replica1] o.a.s.s.ManagedIndexSchema Managed schema znode at /configs/solrCloudCollectionConfig/managed-schema already exists - no need to create it
[junit4] 2> 2793115 INFO (qtp1629588653-30305) [n:127.0.0.1:39480_solr c:test_col s:shard1 x:test_col_shard1_replica1] o.a.s.s.ManagedIndexSchemaFactory After upgrading to managed schema in ZooKeeper, the non-managed schema /configs/solrCloudCollectionConfig/schema15.xml no longer exists.
[junit4] 2> 2793116 INFO (qtp1629588653-30305) [n:127.0.0.1:39480_solr c:test_col s:shard1 x:test_col_shard1_replica1] o.a.s.c.CoreContainer Creating SolrCore 'test_col_shard1_replica1' using configuration from collection test_col
[junit4] 2> 2793116 INFO (qtp1629588653-30305) [n:127.0.0.1:39480_solr c:test_col s:shard1 x:test_col_shard1_replica1] o.a.s.c.SolrCore [[test_col_shard1_replica1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_E880CF9BE662A713-001/tempDir-001/node2/test_col_shard1_replica1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_E880CF9BE662A713-001/tempDir-001/node2/./test_col_shard1_replica1/data/]
[junit4] 2> 2793116 INFO (qtp1629588653-30305) [n:127.0.0.1:39480_solr c:test_col s:shard1 x:test_col_shard1_replica1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3774df79
[junit4] 2> 2793118 INFO (qtp1629588653-30305) [n:127.0.0.1:39480_solr c:test_col s:shard1 x:test_col_shard1_replica1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=25, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=9.767578125, floorSegmentMB=0.3857421875, forceMergeDeletesPctAllowed=11.065629076629586, segmentsPerTier=30.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 2793125 WARN (qtp544137359-30328) [n:127.0.0.1:55101_solr c:test_col s:shard1 x:test_col_shard1_replica2] o.a.s.s.IndexSchema [test_col_shard1_replica2] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 2793132 INFO (qtp1571178598-30325) [n:127.0.0.1:33012_solr c:test_col s:shard3 x:test_col_shard3_replica2] o.a.s.s.ManagedIndexSchema Managed schema znode at /configs/solrCloudCollectionConfig/managed-schema already exists - no need to create it
[junit4] 2> 2793133 INFO (qtp1571178598-30325) [n:127.0.0.1:33012_solr c:test_col s:shard3 x:test_col_shard3_replica2] o.a.s.s.ManagedIndexSchemaFactory After upgrading to managed schema in ZooKeeper, the non-managed schema /configs/solrCloudCollectionConfig/schema15.xml no longer exists.
[junit4] 2> 2793134 INFO (qtp1571178598-30325) [n:127.0.0.1:33012_solr c:test_col s:shard3 x:test_col_shard3_replica2] o.a.s.c.CoreContainer Creating SolrCore 'test_col_shard3_replica2' using configuration from collection test_col
[junit4] 2> 2793135 INFO (qtp1571178598-30325) [n:127.0.0.1:33012_solr c:test_col s:shard3 x:test_col_shard3_replica2] o.a.s.c.SolrCore [[test_col_shard3_replica2] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_E880CF9BE662A713-001/tempDir-001/node7/test_col_shard3_replica2], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_E880CF9BE662A713-001/tempDir-001/node7/./test_col_shard3_replica2/data/]
[junit4] 2> 2793135 INFO (qtp1571178598-30325) [n:127.0.0.1:33012_solr c:test_col s:shard3 x:test_col_shard3_replica2] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3774df79
[junit4] 2> 2793138 INFO (qtp1571178598-30325) [n:127.0.0.1:33012_solr c:test_col s:shard3 x:test_col_shard3_replica2] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=25, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=9.767578125, floorSegmentMB=0.3857421875, forceMergeDeletesPctAllowed=11.065629076629586, segmentsPerTier=30.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 2793146 INFO (qtp544137359-30328) [n:127.0.0.1:55101_solr c:test_col s:shard1 x:test_col_shard1_replica2] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 2793165 WARN (qtp1676013452-30357) [n:127.0.0.1:41793_solr c:test_col s:shard2 x:test_col_shard2_replica2] o.a.s.s.IndexSchema [test_col_shard2_replica2] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 2793186 INFO (qtp544137359-30328) [n:127.0.0.1:55101_solr c:test_col s:shard1 x:test_col_shard1_replica2] o.a.s.s.ManagedIndexSchema Managed schema znode at /configs/solrCloudCollectionConfig/managed-schema already exists - no need to create it
[junit4] 2> 2793187 INFO (qtp544137359-30328) [n:127.0.0.1:55101_solr c:test_col s:shard1 x:test_col_shard1_replica2] o.a.s.s.ManagedIndexSchemaFactory After upgrading to managed schema in ZooKeeper, the non-managed schema /configs/solrCloudCollectionConfig/schema15.xml no longer exists.
[junit4] 2> 2793188 INFO (qtp544137359-30328) [n:127.0.0.1:55101_solr c:test_col s:shard1 x:test_col_shard1_replica2] o.a.s.c.CoreContainer Creating SolrCore 'test_col_shard1_replica2' using configuration from collection test_col
[junit4] 2> 2793189 INFO (qtp544137359-30328) [n:127.0.0.1:55101_solr c:test_col s:shard1 x:test_col_shard1_replica2] o.a.s.c.SolrCore [[test_col_shard1_replica2] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_E880CF9BE662A713-001/tempDir-001/node1/test_col_shard1_replica2], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_E880CF9BE662A713-001/tempDir-001/node1/./test_col_shard1_replica2/data/]
[junit4] 2> 2793189 INFO (qtp544137359-30328) [n:127.0.0.1:55101_solr c:test_col s:shard1 x:test_col_shard1_replica2] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3774df79
[junit4] 2> 2793190 INFO (qtp544137359-30328) [n:127.0.0.1:55101_solr c:test_col s:shard1 x:test_col_shard1_replica2] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=25, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=9.767578125, floorSegmentMB=0.3857421875, forceMergeDeletesPctAllowed=11.065629076629586, segmentsPerTier=30.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 2793217 INFO (qtp1964210261-30310) [n:127.0.0.1:59066_solr c:test_col s:shard3 x:test_col_shard3_replica1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 2793223 INFO (qtp1676013452-30357) [n:127.0.0.1:41793_solr c:test_col s:shard2 x:test_col_shard2_replica2] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 2793243 INFO (qtp1964210261-30310) [n:127.0.0.1:59066_solr c:test_col s:shard3 x:test_col_shard3_replica1] o.a.s.s.ManagedIndexSchema Managed schema znode at /configs/solrCloudCollectionConfig/managed-schema already exists - no need to create it
[junit4] 2> 2793243 INFO (qtp1964210261-30310) [n:127.0.0.1:59066_solr c:test_col s:shard3 x:test_col_shard3_replica1] o.a.s.s.ManagedIndexSchemaFactory After upgrading to managed schema in ZooKeeper, the non-managed schema /configs/solrCloudCollectionConfig/schema15.xml no longer exists.
[junit4] 2> 2793244 INFO (qtp1964210261-30310) [n:127.0.0.1:59066_solr c:test_col s:shard3 x:test_col_shard3_replica1] o.a.s.c.CoreContainer Creating SolrCore 'test_col_shard3_replica1' using configuration from collection test_col
[junit4] 2> 2793245 INFO (qtp1964210261-30310) [n:127.0.0.1:59066_solr c:test_col s:shard3 x:test_col_shard3_replica1] o.a.s.c.SolrCore [[test_col_shard3_replica1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_E880CF9BE662A713-001/tempDir-001/node4/test_col_shard3_replica1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_E880CF9BE662A713-001/tempDir-001/node4/./test_col_shard3_replica1/data/]
[junit4] 2> 2793245 INFO (qtp1964210261-30310) [n:127.0.0.1:59066_solr c:test_col s:shard3 x:test_col_shard3_replica1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3774df79
[junit4] 2> 2793247 INFO (qtp1964210261-30310) [n:127.0.0.1:59066_solr c:test_col s:shard3 x:test_col_shard3_replica1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=25, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=9.767578125, floorSegmentMB=0.3857421875, forceMergeDeletesPctAllowed=11.065629076629586, segmentsPerTier=30.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 2793250 INFO (qtp1676013452-30357) [n:127.0.0.1:41793_solr c:test_col s:shard2 x:test_col_shard2_replica2] o.a.s.s.ManagedIndexSchema Managed schema znode at /configs/solrCloudCollectionConfig/managed-schema already exists - no need to create it
[junit4] 2> 2793250 INFO (qtp1676013452-30357) [n:127.0.0.1:41793_solr c:test_col s:shard2 x:test_col_shard2_replica2] o.a.s.s.ManagedIndexSchemaFactory After upgrading to managed schema in ZooKeeper, the non-managed schema /configs/solrCloudCollectionConfig/schema15.xml no longer exists.
[junit4] 2> 2793251 INFO (qtp1676013452-30357) [n:127.0.0.1:41793_solr c:test_col s:shard2 x:test_col_shard2_replica2] o.a.s.c.CoreContainer Creating SolrCore 'test_col_shard2_replica2' using configuration from collection test_col
[junit4] 2> 2793252 INFO (qtp1676013452-30357) [n:127.0.0.1:41793_solr c:test_col s:shard2 x:test_col_shard2_replica2] o.a.s.c.SolrCore [[test_col_shard2_replica2] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_E880CF9BE662A713-001/tempDir-001/node6/test_col_shard2_replica2], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.TestTolerantUpdateProcessorRandomCloud_E880CF9BE662A713-001/tempDir-001/node6/./test_col_shard2_replica2/data/]
[junit4] 2> 2793252 INFO (qtp1676013452-30357) [n:127.0.0.1:41793_solr c:test_col s:shard2 x:test_col_shard2_replica2] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3774df79
[junit4] 2> 2793253 INFO (qtp1676013452-30357) [n:127.0.0.1:41793_solr c:test_col s:shard2 x:test_col_shard2_replica2] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=25, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=9.767578125, floorSegmentMB=0.3857421875, forceMergeDeletesPctAllowed=11.065629076629586, segmentsPerTier=30.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 2793326 INFO (qtp1964210261-30310) [n:127.0.0.1:59066_solr c:test_col s:shard3 x:test_col_shard3_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2793326 INFO (qtp1964210261-30310) [n:127.0.0.1:59066_solr c:test_col s:shard3 x:test_col_shard3_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2793327 INFO (qtp1964210261-30310) [n:127.0.0.1:59066_solr c:test_col s:shard3 x:test_col_shard3_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2793327 INFO (qtp1964210261-30310) [n:127.0.0.1:59066_solr c:test_col s:shard3 x:test_col_shard3_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2793327 INFO (qtp1964210261-30310) [n:127.0.0.1:59066_solr c:test_col s:shard3 x:test_col_shard3_replica1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=24, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.306980825872113]
[junit4] 2> 2793328 INFO (qtp1964210261-30310) [n:127.0.0.1:59066_solr c:test_col s:shard3 x:test_col_shard3_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@12d02b42[test_col_shard3_replica1] main]
[junit4] 2> 2793336 INFO (qtp544137359-30328) [n:127.0.0.1:55101_solr c:test_col s:shard1 x:test_col_shard1_replica2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2793336 INFO (qtp544137359-30328) [n:127.0.0.1:55101_solr c:test_col s:shard1 x:test_col_shard1_replica2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2793337 INFO (qtp544137359-30328) [n:127.0.0.1:55101_solr c:test_col s:shard1 x:test_col_shard1_replica2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2793337 INFO (qtp544137359-30328) [n:127.0.0.1:55101_solr c:test_col s:shard1 x:test_col_shard1_replica2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2793337 INFO (qtp544137359-30328) [n:127.0.0.1:55101_solr c:test_col s:shard1 x:test_col_shard1_replica2] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=24, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.306980825872113]
[junit4] 2> 2793338 INFO (qtp544137359-30328) [n:127.0.0.1:55101_solr c:test_col s:shard1 x:test_col_shard1_replica2] o.a.s.s.SolrIndexSearcher Opening [Searcher@76a43748[test_col_shard1_replica2] main]
[junit4] 2> 2793340 INFO (qtp544137359-30328) [n:127.0.0.1:55101_solr c:test_col s:shard1 x:test_col_shard1_replica2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/solrCloudCollectionConfig
[junit4] 2> 2793340 INFO (qtp544137359-30328) [n:127.0.0.1:55101_solr c:test_col s:shard1 x:test_col_shard1_replica2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
[junit4] 2> 2793340 INFO (qtp544137359-30328) [n:127.0.0.1:55101_solr c:test_col s:shard1 x:test_col_shard1_replica2] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/solrCloudCollectionConfig/managed-schema
[junit4] 2> 2793341 INFO (qtp544137359-30328) [n:127.0.0.1:55101_solr c:test_col s:shard1 x:test_col_shard1_replica2] o.a.s.s.ZkIndexSchemaReader Retrieved schema version 0 from ZooKeeper
[junit4] 2> 2793345 INFO (qtp1571178598-30325) [n:127.0.0.1:33012_solr c:test_col s:shard3 x:test_col_shard3_replica2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2793345 INFO (qtp1571178598-30325) [n:127.0.0.1:33012_solr c:test_col s:shard3 x:test_col_shard3_replica2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2793345 INFO (qtp544137359-30328) [n:127.0.0.1:55101_solr c:test_col s:shard1 x:test_col_shard1_replica2] o.a.s.s.IndexSchema [test_col_shard1_replica2] Schema name=test
[junit4] 2> 2793346 INFO (qtp1571178598-30325) [n:127.0.0.1:33012_solr c:test_col s:shard3 x:test_col_shard3_replica2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2793346 INFO (qtp1571178598-30325) [n:127.0.0.1:33012_solr c:test_col s:shard3 x:test_col_shard3_replica2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2793347 INFO (qtp1571178598-30325) [n:127.0.0.1:33012_solr c:test_col s:shard3 x:test_col_shard3_replica2] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=24, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.306980825872113]
[junit4] 2> 2793347 INFO (qtp1571178598-30325) [n:127.0.0.1:33012_solr c:test_col s:shard3 x:test_col_shard3_replica2] o.a.s.s.SolrIndexSearcher Opening [Searcher@44f310d4[test_col_shard3_replica2] main]
[junit4] 2> 2793348 INFO (qtp1964210261-30310) [n:127.0.0.1:59066_solr c:test_col s:shard3 x:test_col_shard3_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/solrCloudCollectionConfig
[junit4] 2> 2793349 INFO (qtp1964210261-30310) [n:127.0.0.1:59066_solr c:test_col s:shard3 x:test_col_shard3_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
[junit4] 2> 2793349 INFO (qtp1964210261-30310) [n:127.0.0.1:59066_solr c:test_col s:shard3 x:test_col_shard3_replica1] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/solrCloudCollectionConfig/managed-schema
[junit4] 2> 2793349 INFO (qtp1571178598-30325) [n:127.0.0.1:33012_solr c:test_col s:shard3 x:test_col_shard3_replica2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/solrCloudCollectionConfig
[junit4] 2> 2793349 INFO (qtp1571178598-30325) [n:127.0.0.1:33012_solr c:test_col s:shard3 x:test_col_shard3_replica2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
[junit4] 2> 2793349 INFO (qtp1571178598-30325) [n:127.0.0.1:33012_solr c:test_col s:shard3 x:test_col_shard3_replica2] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/solrCloudCollectionConfig/managed-schema
[junit4] 2> 2793350 INFO (qtp1964210261-30310) [n:127.0.0.1:59066_solr c:test_col s:shard3 x:test_col_shard3_replica1] o.a.s.s.ZkIndexSchemaReader Retrieved schema version 0 from ZooKeeper
[junit4] 2> 2793351 INFO (qtp1438662574-30312) [n:127.0.0.1:61589_solr c:test_col s:shard2 x:test_col_shard2_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2793351 INFO (qtp1438662574-30312) [n:127.0.0.1:61589_solr c:test_col s:shard2 x:test_col_shard2_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2793391 INFO (qtp1438662574-30312) [n:127.0.0.1:61589_solr c:test_col s:shard2 x:test_col_shard2_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2793391 INFO (qtp1438662574-30312) [n:127.0.0.1:61589_solr c:test_col s:shard2 x:test_col_shard2_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2793391 INFO (qtp1438662574-30312) [n:127.0.0.1:61589_solr c:test_col s:shard2 x:test_col_shard2_replica1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=24, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.306980825872113]
[junit4] 2> 2793392 INFO (qtp1438662574-30312) [n:127.0.0.1:61589_solr c:test_col s:shard2 x:test_col_shard2_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@68ca2cfe[test_col_shard2_replica1] main]
[junit4] 2> 2793393 INFO (qtp1438662574-30312) [n:127.0.0.1:61589_solr c:test_col s:shard2 x:test_col_shard2_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/solrCloudCollectionConfig
[junit4] 2> 2793394 INFO (qtp1438662574-30312) [n:127.0.0.1:61589_solr c:test_col s:shard2 x:test_col_shard2_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
[junit4] 2> 2793394 INFO (qtp1438662574-30312) [n:127.0.0.1:61589_solr c:test_col s:shard2 x:test_col_shard2_replica1] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/solrCloudCollectionConfig/managed-schema
[junit4] 2> 2793395 INFO (qtp1438662574-30312) [n:127.0.0.1:61589_solr c:test_col s:shard2 x:test_col_shard2_replica1] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 2793396 INFO (searcherExecutor-11199-thread-1-processing-n:127.0.0.1:61589_solr x:test_col_shard2_replica1 s:shard2 c:test_col) [n:127.0.0.1:61589_solr c:test_col s:shard2 x:test_col_shard2_replica1] o.a.s.c.SolrCore [test_col_shard2_replica1] Registered new searcher Searcher@68ca2cfe[test_col_shard2_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2793396 INFO (qtp1438662574-30312) [n:127.0.0.1:61589_solr c:test_col s:shard2 x:test_col_shard2_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559097833491005440
[junit4] 2> 2793414 INFO (qtp1438662574-30312) [n:127.0.0.1:61589_solr c:test_col s:shard2 x:test_col_shard2_replica1] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard2: total=2 found=1 timeoutin=9992ms
[junit4] 2> 2793414 INFO (zkCallback-5495-thread-1-processing-n:127.0.0.1:33012_solr) [n:127.0.0.1:33012_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [7])
[junit4] 2> 2793414 INFO (zkCallback-5497-thread-2-processing-n:127.0.0.1:61589_solr) [n:127.0.0.1:61589_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [7])
[junit4] 2> 2793415 INFO (zkCallback-5494-thread-1-processing-n:127.0.0.1:55101_solr) [n:127.0.0.1:55101_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [7])
[junit4] 2> 2793414 INFO (zkCallback-5496-thread-2-processing-n:127.0.0.1:59066_solr) [n:127.0.0.1:59066_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [7])
[junit4] 2> 2793415 INFO (zkCallback-5490-thread-1-processing-n:127.0.0.1:39480_solr) [n:127.0.0.1:39480_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [7])
[junit4] 2> 2793416 INFO (zkCallback-5488-thread-1-processing-n:127.0.0.1:41793_solr) [n:127.0.0.1:41793_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [7])
[junit4] 2> 2793426 INFO (qtp1571178598-30325) [n:127.0.0.1:33012_solr c:test_col s:shard3 x:test_col_shard3_replica2] o.a.s.s.ZkIndexSchemaReader Retrieved schema version 0 from ZooKeeper
[junit4] 2> 2793427 INFO (qtp1964210261-30310) [n:127.0.0.1:59066_solr c:test_col s:shard3 x:test_col_shard3_replica1] o.a.s.s.IndexSchema [test_col_shard3_replica1] Schema name=test
[junit4] 2> 2793429 INFO (qtp1571178598-30325) [n:127.0.0.1:33012_solr c:test_col s:shard3 x:test_col_shard3_replica2] o.a.s.s.IndexSchema [test_col_shard3_replica2] Schema name=test
[junit4] 2> 2793474 INFO (qtp1676013452-30357) [n:127.0.0.1:41793_solr c:test_col s:shard2 x:test_col_shard2_replica2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2793474 INFO (qtp1676013452-30357) [n:127.0.0.1:41793_solr c:test_col s:shard2 x:test_col_shard2_replica2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2793475 INFO (qtp1676013452-30357) [n:127.0.0.1:41793_solr c:test_col s:shard2 x:test_col_shard2_replica2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2793475 INFO (qtp1676013452-30357) [n:127.0.0.1:41793_solr c:test_col s:shard2 x:test_col_shard2_replica2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2793476 INFO (qtp1676013452-30357) [n:127.0.0.1:41793_solr c:test_col s:shard2 x:test_col_shard2_replica2] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=24, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.306980825872113]
[junit4] 2> 2793477 INFO (qtp1676013452-30357) [n:127.0.0.1:41793_solr c:test_col s:shard2 x:test_col_shard2_replica2] o.a.s.s.SolrIndexSearcher Opening [Searcher@3075adde[test_col_shard2_replica2] main]
[junit4] 2> 2793478 INFO (qtp1676013452-30357) [n:127.0.0.1:41793_solr c:test_col s:shard2 x:test_col_shard2_replica2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/solrCloudCollectionConfig
[junit4] 2> 2793479 INFO (qtp1676013452-30357) [n:127.0.0.1:41793_solr c:test_col s:shard2 x:test_col_shard2_replica2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
[junit4] 2> 2793479 INFO (qtp1676013452-30357) [n:127.0.0.1:41793_solr c:test_col s:shard2 x:test_col_shard2_replica2] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/solrCloudCollectionConfig/managed-schema
[junit4] 2> 2793480 INFO (qtp1676013452-30357) [n:127.0.0.1:41793_solr c:test_col s:shard2 x:test_col_shard2_replica2] o.a.s.s.ZkIndexSchemaReader Retrieved schema version 0 from ZooKeeper
[junit4] 2> 2793507 INFO (qtp1676013452-30357) [n:127.0.0.1:41793_solr c:test_col s:shard2 x:test_col_shard2_replica2] o.a.s.s.IndexSchema [test_col_shard2_replica2] Schema name=test
[junit4] 2> 2793518 WARN (qtp544137359-30328) [n:127.0.0.1:55101_solr c:test_col s:shard1 x:test_col_shard1_replica2] o.a.s.s.IndexSchema [test_col_shard1_replica2] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 2793544 WARN (qtp1964210261-30310) [n:127.0.0.1:59066_solr c:test_col s:shard3 x:test_col_shard3_replica1] o.a.s.s.IndexSchema [test_col_shard3_replica1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 2793580 INFO (qtp1629588653-30305) [n:127.0.0.1:39480_solr c:test_col s:shard1 x:test_col_shard1_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2793581 INFO (qtp1629588653-30305) [n:127.0.0.1:39480_solr c:test_col s:shard1 x:test_col_shard1_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2793593 INFO (qtp1629588653-30305) [n:127.0.0.1:39480_solr c:test_col s:shard1 x:test_col_shard1_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2793593 INFO (qtp1629588653-30305) [n:127.0.0.1:39480_solr c:test_col s:shard1 x:test_col_shard1_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2793594 INFO (qtp1629588653-30305) [n:127.0.0.1:39480_solr c:test_col s:shard1 x:test_col_shard1_replica1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=24, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.306980825872113]
[junit4] 2> 2793594 INFO (qtp1629588653-30305) [n:127.0.0.1:39480_solr c:test_col s:shard1 x:test_col_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@f3778d0[test_col_shard1_replica1] main]
[junit4] 2> 2793623 WARN (qtp1676013452-30357) [n:127.0.0.1:41793_solr c:test_col s:shard2 x:test_col_shard2_replica2] o.a.s.s.IndexSchema [test_col_shard2_replica2] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 2793629 WARN (qtp1571178598-30325) [n:127.0.0.1:33012_solr c:test_col s:shard3 x:test_col_shard3_replica2] o.a.s.s.IndexSchema [test_col_shard3_replica2] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 2793653 INFO (qtp1676013452-30357) [n:127.0.0.1:41793_solr c:test_col s:shard2 x:test_col_shard2_replica2] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 2793654 INFO (qtp544137359-30328) [n:127.0.0.1:55101_solr c:test_col s:shard1 x:test_col_shard1_replica2] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 2793655 INFO (qtp1964210261-30310) [n:127.0.0.1:59066_solr c:test_col s:shard3 x:test_col_shard3_replica1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 2793666 INFO (qtp1571178598-30325) [n:127.0.0.1:33012_solr c:test_col s:shard3 x:test_col_shard3_replica2] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 2793677 INFO (qtp1676013452-30357) [n:127.0.0.1:41793_solr c:test_col s:shard2 x:test_col_shard2_replica2] o.a.s.s.ZkIndexSchemaReader Finished refreshing schema in 196 ms
[junit4] 2> 2793678 INFO (searcherExecutor-11204-thread-1-processing-n:127.0.0.1:41793_solr x:test_col_shard2_replica2 s:shard2 c:test_col) [n:127.0.0.1:41793_solr c:test_col s:shard2 x:test_col_shard2_replica2] o.a.s.c.SolrCore [test_col_shard2_replica2] Registered new searcher Searcher@3075adde[test_col_shard2_replica2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2793679 INFO (qtp1676013452-30357) [n:127.0.0.1:41793_solr c:test_col s:shard2 x:test_col_shard2_replica2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559097833787752448
[junit4] 2> 2793689 INFO (qtp1629588653-30305) [n:127.0.0.1:39480_solr c:test_col s:shard1 x:test_col_shard1_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/solrCloudCollectionConfig
[junit4] 2> 2793691 INFO (qtp1629588653-30305) [n:127.0.0.1:39480_solr c:test_col s:shard1 x:test_col_shard1_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/solrCloudCollectionConfig
[junit4] 2> 2793691 INFO (qtp1629588653-30305) [n:127.0.0.1:39480_solr c:test_col s:shard1 x:test_col_shard1_replica1] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/solrCloudCollectionConfig/managed-schema
[junit4] 2> 2793697 INFO (qtp1629588653-30305) [n:127.0.0.1:39480_solr c:test_col s:shard1 x:test_col_shard1_replica1] o.a.s.s.ZkIndexSchemaReader Retrieved schema version 0 from ZooKeeper
[junit4] 2> 2793701 INFO (qtp1629588653-30305) [n:127.0.0.1:39480_solr c:test_col s:shard1 x:test_col_shard1_replica1] o.a.s.s.IndexSchema [test_col_shard1_replica1] Schema name=test
[junit4] 2> 2793756 INFO (qtp544137359-30328) [n:127.0.0.1:55101_solr c:test_col s:shard1 x:test_col_shard1_replica2] o.a.s.s.ZkIndexSchemaReader Finished refreshing schema in 414 ms
[junit4] 2> 2793757 INFO (qtp1964210261-30310) [n:127.0.0.1:59066_solr c:test_col s:shard3 x:test_col_shard3_replica1] o.a.s.s.ZkIndexSchemaReader Finished refreshing schema in 407 ms
[junit4] 2> 2793757 INFO (searcherExecutor-11202-thread-1-processing-n:127.0.0.1:55101_solr x:test_col_shard1_replica2 s:shard1 c:test_col) [n:127.0.0.1:55101_solr c:test_col s:shard1 x:test_col_shard1_replica2] o.a.s.c.SolrCore [test_col_shard1_replica2] Registered new searcher Searcher@76a43748[test_col_shard1_replica2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2793758 INFO (qtp544137359-30328) [n:127.0.0.1:55101_solr c:test_col s:shard1 x:test_col_shard1_replica2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559097833870589952
[junit4] 2> 2793758 INFO (qtp1964210261-30310) [n:127.0.0.1:59066_solr c:test_col s:shard3 x:test_col_shard3_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559097833870589952
[junit4] 2> 2793760 INFO (searcherExecutor-11203-thread-1-processing-n:127.0.0.1:59066_solr x:test_col_shard3_replica1 s:shard3 c:test_col) [n:127.0.0.1:59066_solr c:test_col s:shard3 x:test_col_shard3_replica1] o.a.s.c.SolrCore [test_col_shard3_replica1] Registered new searcher Searcher@12d02b42[test_col_shard3_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2793766 INFO (qtp1571178598-30325) [n:127.0.0.1:33012_solr c:test_col s:shard3 x:test_col_shard3_replica2] o.a.s.s.ZkIndexSchemaReader Finished refreshing schema in 340 ms
[junit4] 2> 2793768 INFO (searcherExecutor-11201-thread-1-processing-n:127.0.0.1:33012_solr x:test_col_shard3_replica2 s:shard3 c:test_col) [n:127.0.0.1:33012_solr c:test_col s:shard3 x:test_col_shard3_replica2] o.a.s.c.SolrCore [test_col_shard3_replica2] Registered new searcher Searcher@44f310d4[test_col_shard3_replica2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2793768 INFO (qtp1571178598-30325) [n:127.0.0.1:33012_solr c:test_col s:shard3 x:test_col_shard3_replica2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559097833881075712
[junit4] 2> 2793769 INFO (qtp544137359-30328) [n:127.0.0.1:55101_solr c:test_col s:shard1 x:test_col_shard1_replica2] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
[junit4] 2> 2793769 INFO (qtp1964210261-30310) [n:127.0.0.1:59066_solr c:test_col s:shard3 x:test_col_shard3_replica1] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard3: total=2 found=1 timeoutin=9999ms
[junit4] 2> 2793772 WARN (qtp1629588653-30305) [n:127.0.0.1:39480_solr c:test_col s:shard1 x:test_col_shard1_replica1] o.a.s.s.IndexSchema [test_col_shard1_replica1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 2793792 INFO (qtp1629588653-30305) [n:127.0.0.1:39480_solr c:test_col s:shard1 x:test_col_shard1_replica1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 2793811 INFO (qtp1629588653-30305) [n:127.0.0.1:39480_solr c:test_col s:shard1 x:test_col_shard1_replica1] o.a.s.s.ZkIndexSchemaReader Finished refreshing schema in 113 ms
[junit4] 2> 2793811 INFO (searcherExecutor-11200-thread-1-processing-n:127.0.0.1:39480_solr x:test_col_shard1_replica1 s:shard1 c:test_col) [n:127.0.0.1:39480_solr c:test_col s:shard1 x:test_col_shard1_replica1] o.a.s.c.SolrCore [test_col_shard1_replica1] Registered new searcher Searcher@f3778d0[test_col_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2793812 INFO (qtp1629588653-30305) [n:127.0.0.1:39480_solr c:test_col s:shard1 x:test_col_shard1_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559097833927213056
[junit4] 2> 2793875 INFO (zkCallback-5490-thread-1-processing-n:127.0.0.1:39480_solr) [n:127.0.0.1:39480_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [7])
[junit4] 2> 2793875 INFO (zkCallback-5494-thread-1-processing-n:127.0.0.1:55101_solr) [n:127.0.0.1:55101_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [7])
[junit4] 2> 2793876 INFO (zkCallback-5496-thread-2-processing-n:127.0.0.1:59066_solr) [n:127.0.0.1:59066_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [7])
[junit4] 2> 2793875 INFO (zkCallback-5488-thread-1-processing-n:127.0.0.1:41793_solr) [n:127.0.0.1:41793_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [7])
[junit4] 2> 2793876 INFO (zkCallback-5497-thread-2-processing-n:127.0.0.1:61589_solr) [n:127.0.0.1:61589_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [7])
[junit4] 2> 2793876 INFO (zkCallback-5495-thread-3-processing-n:127.0.0.1:33012_solr) [n:127.0.0.1:33012_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [7])
[junit4] 2> 2793917 INFO (qtp1438662574-30312) [n:127.0.0.1:61589_solr c:test_col s:shard2 x:test_col_shard2_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2793917 INFO (qtp1438662574-30312) [n:127.0.0.1:61589_solr c:test_col s:shard2 x:test_col_shard2_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2793917 INFO (qtp1438662574-30312) [n:127.0.0.1:61589_solr c:test_col s:shard2 x:test_col_shard2_replica1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:61589/solr/test_col_shard2_replica1/
[junit4] 2> 2793918 INFO (qtp1438662574-30312) [n:127.0.0.1:61589_solr c:test_col s:shard2 x:test_col_shard2_replica1] o.a.s.u.PeerSync PeerSync: core=test_col_shard2_replica1 url=http://127.0.0.1:61589/solr START replicas=[http://127.0.0.1:41793/solr/test_col_shard2_replica2/] nUpdates=100
[junit4] 2> 2793918 INFO (qtp1438662574-30312) [n:127.0.0.1:61589_solr c:test_col s:shard2 x:test_col_shard2_replica1] o.a.s.u.PeerSync PeerSync: core=test_col_shard2_replica1 url=http://127.0.0.1:61589/solr DONE. We have no versions. sync failed.
[junit4] 2> 2793919 INFO (qtp1676013452-30359) [n:127.0.0.1:41793_solr c:test_col s:shard2 r:core_node2 x:test_col_shard2_replica2] o.a.s.c.S.Request [test_col_shard2_replica2] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2793920 INFO (qtp1438662574-30312) [n:127.0.0.1:61589_solr c:test_col s:shard2 x:test_col_shard2_replica1] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 2793920 INFO (qtp1438662574-30312) [n:127.0.0.1:61589_solr c:test_col s:shard2 x:test_col_shard2_replica1] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 2793926 INFO (qtp1438662574-30312) [n:127.0.0.1:61589_solr c:test_col s:shard2 x:test_col_shard2_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:61589/solr/test_col_shard2_replica1/ shard2
[junit4] 2> 2794031 INFO (zkCallback-5490-thread-1-processing-n:127.0.0.1:39480_solr) [n:127.0.0.1:39480_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [7])
[junit4] 2> 2794031 INFO (zkCallback-5488-thread-1-processing-n:127.0.0.1:41793_solr) [n:127.0.0.1:41793_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [7])
[junit4] 2> 2794031 INFO (zkCallback-5494-thread-1-processing-n:127.0.0.1:55101_solr) [n:127.0.0.1:55101_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [7])
[junit4] 2> 2794031 INFO (zkCallback-5496-thread-2-processing-n:127.0.0.1:59066_solr) [n:127.0.0.1:59066_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [7])
[junit4] 2> 2794031 INFO (zkCallback-5497-thread-2-processing-n:127.0.0.1:61589_solr) [n:127.0.0.1:61589_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [7])
[junit4] 2> 2794031 INFO (zkCallback-5495-thread-3-processing-n:127.0.0.1:33012_solr) [n:127.0.0.1:33012_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [7])
[junit4] 2> 2794082 INFO (qtp1438662574-30312) [n:127.0.0.1:61589_solr c:test_col s:shard2 x:test_col_shard2_replica1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2794085 INFO (qtp1438662574-30312) [n:127.0.0.1:61589_solr c:test_col s:shard2 x:test_col_shard2_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={property.config=solrconfig-distrib-update-processor-chains.xml&property.schema=schema15.xml&qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&name=test_col_shard2_replica1&action=CREATE&numShards=3&collection=test_col&shard=shard2&wt=javabin&version=2} status=0 QTime=2400
[junit4] 2> 2794189 INFO (zkCallback-5490-thread-1-processing-n:127.0.0.1:39480_solr) [n:127.0.0.1:39480_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [7])
[junit4] 2> 2794189 INFO (zkCallback-5488-thread-1-processing-n:127.0.0.1:41793_solr) [n:127.0.0.1:41793_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [7])
[junit4] 2> 2794189 INFO (zkCallback-5494-thread-1-processing-n:127.0.0.1:55101_solr) [n:127.0.0.1:55101_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [7])
[junit4] 2> 2794189 INFO (zkCallback-5496-thread-2-processing-n:127.0.0.1:59066_solr) [n:127.0.0.1:59066_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [7])
[junit4] 2> 2794190 INFO (zkCallback-5495-thread-2-processing-n:127.0.0.1:33012_solr) [n:127.0.0.1:33012_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [7])
[junit4] 2> 2794191 INFO (zkCallback-5497-thread-2-processing-n:127.0.0.1:61589_solr) [n:127.0.0.1:61589_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/test_col/state.json] for collection [test_col] has occurred - updating... (live nodes size: [7])
[junit4] 2> 2794277 INFO (qtp1964210261-30310) [n:127.0.0.1:59066_solr c:test_col s:shard3 x:test_col_shard3_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2794277 INFO (qtp1964210261-30310) [n:127.0.0.1:59066_solr c:test_col s:shard3 x:test_col_shard3_replica1] o.a.s.c.
[...truncated too long message...]
NumericTerms32, DistributedDebugComponentTest, TestRecoveryHdfs, TestFuzzyAnalyzedSuggestions, DistributedFacetPivotLongTailTest, TestManagedSchema, TestMiniSolrCloudCluster, TestPKIAuthenticationPlugin, TestSolrConfigHandlerCloud, TestStressRecovery, JsonValidatorTest, TestConfigsApi, TestSuggestSpellingConverter, TestIndexingPerformance, StatsComponentTest, BadIndexSchemaTest, SolrCoreMetricManagerTest, TestLeaderElectionZkExpiry, ChangedSchemaMergeTest, TestNoOpRegenerator, WordBreakSolrSpellCheckerTest, MinimalSchemaTest, TestRangeQuery, SolrGangliaReporterTest, TestConfig, BasicDistributedZk2Test, AnalysisErrorHandlingTest, TestHdfsCloudBackupRestore, URLClassifyProcessorTest, TestFieldSortValues, SuggesterTest, TestOmitPositions, TestMiniSolrCloudClusterSSL, FullHLLTest, TestSSLRandomization, TestDistributedGrouping, HighlighterConfigTest, TestClusterStateMutator, TestQueryWrapperFilter, LeaderInitiatedRecoveryOnCommitTest, TestManagedStopFilterFactory, TestQueryUtils, FileBasedSpellCheckerTest, FieldMutatingUpdateProcessorTest, SmileWriterTest, TestSolrCoreParser, ExternalFileFieldSortTest, TestCloudSchemaless, HLLUtilTest, CacheHeaderTest, DistributedIntervalFacetingTest, TestJsonFacets, DistributedFacetExistsSmallTest, BlockCacheTest, TestMaxScoreQueryParser, TestNonDefinedSimilarityFactory, TestLocalFSCloudBackupRestore, TestReload, IndexSchemaTest, ConjunctionSolrSpellCheckerTest, TestSubQueryTransformer, BadCopyFieldTest, VMParamsZkACLAndCredentialsProvidersTest, CustomCollectionTest, TestXmlQParserPlugin, CSVRequestHandlerTest, SortByFunctionTest, LeaderFailoverAfterPartitionTest, PKIAuthenticationIntegrationTest, DistributedQueryComponentCustomSortTest, TestSolrCoreProperties, BasicZkTest, DocValuesMissingTest, TestTrieFacet, TestFieldCacheSortRandom, TestQueryTypes, TestCrossCoreJoin, DeleteReplicaTest, SuggestComponentContextFilterQueryTest, ReplicationFactorTest, TestDistributedMissingSort, BasicFunctionalityTest, CurrencyFieldOpenExchangeTest, TestMacros, SimpleCollectionCreateDeleteTest, TestInitParams, TestShortCircuitedRequests, TestRebalanceLeaders, BitVectorTest, TestCopyFieldCollectionResource, RegexBoostProcessorTest, TestSolrQueryResponse, TestPathTrie, DistributedFacetPivotSmallTest, FullSolrCloudDistribCmdsTest, CursorMarkTest, TestSQLHandler, RankQueryTest, ZkStateWriterTest, TestClusterProperties, HdfsRecoveryZkTest, ShardRoutingCustomTest, SolrCoreCheckLockOnStartupTest, BJQParserTest, OverriddenZkACLAndCredentialsProvidersTest, TestCloudManagedSchema, TestSolrCloudWithSecureImpersonation, TestCloudInspectUtil, TestSolrCloudWithDelegationTokens, TestLuceneMatchVersion, TestReloadAndDeleteDocs, SuggesterWFSTTest, SpellPossibilityIteratorTest, TestSolr4Spatial, TestRandomFaceting, MetricsHandlerTest, TestUseDocValuesAsStored, TestExceedMaxTermLength, TestInPlaceUpdatesDistrib, TlogReplayBufferedWhileIndexingTest, TestPHPSerializedResponseWriter, TestSearcherReuse, TestRTimerTree, ScriptEngineTest, CloudMLTQParserTest, ConnectionManagerTest, TestDynamicFieldCollectionResource, SolrIndexSplitterTest, TestScoreJoinQPScore, HdfsChaosMonkeySafeLeaderTest, DistributedTermsComponentTest, TestCloudPseudoReturnFields, TestHdfsBackupRestoreCore, TestSolrConfigHandler, CachingDirectoryFactoryTest, DebugComponentTest, MultiThreadedOCPTest, TestNamedUpdateProcessors, ManagedSchemaRoundRobinCloudTest, CloudExitableDirectoryReaderTest, TestIntervalFaceting, TestSolrIndexConfig, TestTrie, TestManagedSchemaThreadSafety, DataDrivenBlockJoinTest, TestSolrQueryParserResource, DirectUpdateHandlerOptimizeTest, TestUninvertingReader, OpenCloseCoreStressTest, LeaderElectionIntegrationTest, RecoveryZkTest, ZkSolrClientTest, TestRandomDVFaceting, TestDistributedSearch, TestHashPartitioner, TestJoin, TestCoreContainer, SimpleFacetsTest, SolrCoreTest, SolrCmdDistributorTest, TestFiltering, TestLazyCores, TestIndexSearcher, DistributedQueryElevationComponentTest, SolrRequestParserTest, SuggesterTSTTest, PolyFieldTest, NoCacheHeaderTest, SchemaVersionSpecificBehaviorTest, DefaultValueUpdateProcessorTest, StatelessScriptUpdateProcessorFactoryTest, TestRemoteStreaming, DistanceFunctionTest, TestSolrDeletionPolicy1, SolrInfoMBeanTest, IndexBasedSpellCheckerTest, TestSurroundQueryParser, DisMaxRequestHandlerTest, DirectSolrSpellCheckerTest, TestReversedWildcardFilterFactory, PrimitiveFieldTypeTest, TestValueSourceCache, TermVectorComponentTest, FieldAnalysisRequestHandlerTest, IndexSchemaRuntimeFieldTest, LoggingHandlerTest, ReturnFieldsTest, QueryParsingTest, UniqFieldsUpdateProcessorFactoryTest, BinaryUpdateRequestHandlerTest, TestComponentsName, SearchHandlerTest, SOLR749Test, TestQuerySenderListener, UpdateParamsTest, CopyFieldTest, ResponseLogComponentTest, BadComponentTest, MultiTermTest, SampleTest, TestDocSet, TestSearchPerf, OutputWriterTest, DirectSolrConnectionTest, TestPhraseSuggestions, TestCodecSupport, EchoParamsTest, TestSweetSpotSimilarityFactory, TestDFRSimilarityFactory, TestLMDirichletSimilarityFactory, TestLMJelinekMercerSimilarityFactory, TestBM25SimilarityFactory, TestFastWriter, TestJmxMonitoredMap, TimeZoneUtilsTest, TestFastOutputStream, OpenExchangeRatesOrgProviderTest, DateMathParserTest, PreAnalyzedFieldTest, TestSystemIdResolver, SpellingQueryConverterTest, DOMUtilTest, RAMDirectoryFactoryTest, ClusterStateTest, TestSolrJ, TestUtils, ZkNodePropsTest, SliceStateTest, SystemInfoHandlerTest, UUIDFieldTest, CircularListTest, TestRTGBase, CursorPagingTest, CleanupOldIndexTest, CollectionTooManyReplicasTest, CollectionsAPISolrJTest, DeleteShardTest, DeleteStatusTest, DistributedQueueTest, DistributedVersionInfoTest, LeaderElectionContextKeyTest, LeaderInitiatedRecoveryOnShardRestartTest, MigrateRouteKeyTest, MissingSegmentRecoveryTest, OutOfBoxZkACLAndCredentialsProvidersTest, OverseerModifyCollectionTest, OverseerRolesTest, OverseerStatusTest, RemoteQueryErrorTest, RollingRestartTest, SharedFSAutoReplicaFailoverTest, SolrCLIZkUtilsTest, SolrCloudExampleTest, SolrXmlInZkTest, TestAuthenticationFramework, TestCloudDeleteByQuery, TestCollectionAPI, TestConfigSetsAPI, TestConfigSetsAPIZkFailure, TestDeleteCollectionOnDownNodes, TestDownShardTolerantSearch, TestRandomFlRTGCloud, TestReplicaProperties, TestRequestForwarding, TestRequestStatusCollectionAPI, TestSegmentSorting, TestSolrCloudWithKerberosAlt, TestStressCloudBlindAtomicUpdates, TestStressLiveNodes, TestTolerantUpdateProcessorCloud, TestTolerantUpdateProcessorRandomCloud]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestTolerantUpdateProcessorRandomCloud -Dtests.seed=E880CF9BE662A713 -Dtests.slow=true -Dtests.locale=ar-YE -Dtests.timezone=Africa/El_Aaiun -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] ERROR 0.00s J1 | TestTolerantUpdateProcessorRandomCloud (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.TestTolerantUpdateProcessorRandomCloud:
[junit4] > 1) Thread[id=30574, name=OverseerHdfsCoreFailoverThread-97443614124081172-127.0.0.1:59066_solr-n_0000000004, state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:139)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 2) Thread[id=30572, name=OverseerStateUpdate-97443614124081172-127.0.0.1:59066_solr-n_0000000004, state=TIMED_WAITING, group=Overseer state updater.]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
[junit4] > at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:164)
[junit4] > at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:138)
[junit4] > at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:179)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > at __randomizedtesting.SeedInfo.seed([E880CF9BE662A713]:0)
[junit4] Completed [535/694 (1!)] on J1 in 37.04s, 2 tests, 1 error <<< FAILURES!
[...truncated 62711 lines...]
-documentation-lint:
[jtidy] Checking for broken html (such as invalid tags)...
[delete] Deleting directory /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/build/jtidy_tmp
[echo] Checking for broken links...
[exec]
[exec] Crawl/parse...
[exec]
[exec] Verify...
[echo] Checking for malformed docs...
[exec]
[exec] /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/docs/solr-solrj/overview-summary.html
[exec] missing: org.apache.solr.client.solrj.io.eval
[exec]
[exec] Missing javadocs were found!
BUILD FAILED
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/build.xml:775: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/build.xml:101: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build.xml:654: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build.xml:670: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/common-build.xml:2493: exec returned: 1
Total time: 97 minutes 50 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build #
1128 - Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1128/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC
1 tests failed.
FAILED: org.apache.solr.cloud.PeerSyncReplicationTest.test
Error Message:
timeout waiting to see all nodes active
Stack Trace:
java.lang.AssertionError: timeout waiting to see all nodes active
at __randomizedtesting.SeedInfo.seed([E20191D1B2826B5B:6A55AE0B1C7E06A3]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.PeerSyncReplicationTest.waitTillNodesActive(PeerSyncReplicationTest.java:326)
at org.apache.solr.cloud.PeerSyncReplicationTest.bringUpDeadNodeAndEnsureNoReplication(PeerSyncReplicationTest.java:277)
at org.apache.solr.cloud.PeerSyncReplicationTest.forceNodeFailureAndDoPeerSync(PeerSyncReplicationTest.java:259)
at org.apache.solr.cloud.PeerSyncReplicationTest.test(PeerSyncReplicationTest.java:138)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 12470 lines...]
[junit4] Suite: org.apache.solr.cloud.PeerSyncReplicationTest
[junit4] 2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_E20191D1B2826B5B-001/init-core-data-001
[junit4] 2> 2562982 INFO (SUITE-PeerSyncReplicationTest-seed#[E20191D1B2826B5B]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields
[junit4] 2> 2562984 INFO (SUITE-PeerSyncReplicationTest-seed#[E20191D1B2826B5B]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 2562984 INFO (SUITE-PeerSyncReplicationTest-seed#[E20191D1B2826B5B]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /_/qu
[junit4] 2> 2562987 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 2562987 INFO (Thread-7664) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2562987 INFO (Thread-7664) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 2563087 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.c.ZkTestServer start zk server on port:50432
[junit4] 2> 2563104 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 2563107 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 2563110 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 2563112 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 2563114 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 2563116 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 2563118 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 2563119 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 2563121 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 2563123 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 2563125 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 2563267 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_E20191D1B2826B5B-001/control-001/cores/collection1
[junit4] 2> 2563269 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 2563270 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5cf41e13{/_/qu,null,AVAILABLE}
[junit4] 2> 2563270 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@f40ed5{HTTP/1.1,[http/1.1]}{127.0.0.1:58617}
[junit4] 2> 2563270 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.e.j.s.Server Started @2567091ms
[junit4] 2> 2563270 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_E20191D1B2826B5B-001/tempDir-001/control/data, hostContext=/_/qu, hostPort=58617, coreRootDirectory=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_E20191D1B2826B5B-001/control-001/cores}
[junit4] 2> 2563271 ERROR (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2563272 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 2563272 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2563272 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2563272 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-13T04:15:32.450Z
[junit4] 2> 2563276 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2563276 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_E20191D1B2826B5B-001/control-001/solr.xml
[junit4] 2> 2563281 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:50432/solr
[junit4] 2> 2563328 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [n:127.0.0.1:58617__%2Fqu ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2563329 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [n:127.0.0.1:58617__%2Fqu ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:58617__%2Fqu
[junit4] 2> 2563331 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [n:127.0.0.1:58617__%2Fqu ] o.a.s.c.Overseer Overseer (id=97449366792896516-127.0.0.1:58617__%2Fqu-n_0000000000) starting
[junit4] 2> 2563345 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [n:127.0.0.1:58617__%2Fqu ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58617__%2Fqu
[junit4] 2> 2563349 INFO (OverseerStateUpdate-97449366792896516-127.0.0.1:58617__%2Fqu-n_0000000000) [n:127.0.0.1:58617__%2Fqu ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2563444 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [n:127.0.0.1:58617__%2Fqu ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_E20191D1B2826B5B-001/control-001/cores
[junit4] 2> 2563444 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [n:127.0.0.1:58617__%2Fqu ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 2563449 INFO (OverseerStateUpdate-97449366792896516-127.0.0.1:58617__%2Fqu-n_0000000000) [n:127.0.0.1:58617__%2Fqu ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 2563466 INFO (coreLoadExecutor-9137-thread-1-processing-n:127.0.0.1:58617__%2Fqu) [n:127.0.0.1:58617__%2Fqu c:control_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2563479 INFO (coreLoadExecutor-9137-thread-1-processing-n:127.0.0.1:58617__%2Fqu) [n:127.0.0.1:58617__%2Fqu c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 2563570 WARN (coreLoadExecutor-9137-thread-1-processing-n:127.0.0.1:58617__%2Fqu) [n:127.0.0.1:58617__%2Fqu c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 2563572 INFO (coreLoadExecutor-9137-thread-1-processing-n:127.0.0.1:58617__%2Fqu) [n:127.0.0.1:58617__%2Fqu c:control_collection x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 2563588 INFO (coreLoadExecutor-9137-thread-1-processing-n:127.0.0.1:58617__%2Fqu) [n:127.0.0.1:58617__%2Fqu c:control_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 2563588 INFO (coreLoadExecutor-9137-thread-1-processing-n:127.0.0.1:58617__%2Fqu) [n:127.0.0.1:58617__%2Fqu c:control_collection x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_E20191D1B2826B5B-001/control-001/cores/collection1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_E20191D1B2826B5B-001/control-001/cores/collection1/data/]
[junit4] 2> 2563589 INFO (coreLoadExecutor-9137-thread-1-processing-n:127.0.0.1:58617__%2Fqu) [n:127.0.0.1:58617__%2Fqu c:control_collection x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@24d53b22
[junit4] 2> 2563591 INFO (coreLoadExecutor-9137-thread-1-processing-n:127.0.0.1:58617__%2Fqu) [n:127.0.0.1:58617__%2Fqu c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=7, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 2563602 WARN (coreLoadExecutor-9137-thread-1-processing-n:127.0.0.1:58617__%2Fqu) [n:127.0.0.1:58617__%2Fqu c:control_collection x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 2563654 INFO (coreLoadExecutor-9137-thread-1-processing-n:127.0.0.1:58617__%2Fqu) [n:127.0.0.1:58617__%2Fqu c:control_collection x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2563654 INFO (coreLoadExecutor-9137-thread-1-processing-n:127.0.0.1:58617__%2Fqu) [n:127.0.0.1:58617__%2Fqu c:control_collection x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=1000 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2563655 INFO (coreLoadExecutor-9137-thread-1-processing-n:127.0.0.1:58617__%2Fqu) [n:127.0.0.1:58617__%2Fqu c:control_collection x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2563655 INFO (coreLoadExecutor-9137-thread-1-processing-n:127.0.0.1:58617__%2Fqu) [n:127.0.0.1:58617__%2Fqu c:control_collection x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2563656 INFO (coreLoadExecutor-9137-thread-1-processing-n:127.0.0.1:58617__%2Fqu) [n:127.0.0.1:58617__%2Fqu c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=19, maxMergeAtOnceExplicit=29, maxMergedSegmentMB=93.8798828125, floorSegmentMB=0.9091796875, forceMergeDeletesPctAllowed=2.908175163128498, segmentsPerTier=13.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7895201225707248
[junit4] 2> 2563656 INFO (coreLoadExecutor-9137-thread-1-processing-n:127.0.0.1:58617__%2Fqu) [n:127.0.0.1:58617__%2Fqu c:control_collection x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@318a957e[collection1] main]
[junit4] 2> 2563658 INFO (coreLoadExecutor-9137-thread-1-processing-n:127.0.0.1:58617__%2Fqu) [n:127.0.0.1:58617__%2Fqu c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2563658 INFO (coreLoadExecutor-9137-thread-1-processing-n:127.0.0.1:58617__%2Fqu) [n:127.0.0.1:58617__%2Fqu c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2563658 INFO (coreLoadExecutor-9137-thread-1-processing-n:127.0.0.1:58617__%2Fqu) [n:127.0.0.1:58617__%2Fqu c:control_collection x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2563660 INFO (searcherExecutor-9138-thread-1-processing-n:127.0.0.1:58617__%2Fqu x:collection1 c:control_collection) [n:127.0.0.1:58617__%2Fqu c:control_collection x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@318a957e[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2563661 INFO (coreLoadExecutor-9137-thread-1-processing-n:127.0.0.1:58617__%2Fqu) [n:127.0.0.1:58617__%2Fqu c:control_collection x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559189869390987264
[junit4] 2> 2563670 INFO (coreZkRegister-9130-thread-1-processing-n:127.0.0.1:58617__%2Fqu x:collection1 c:control_collection) [n:127.0.0.1:58617__%2Fqu c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2563671 INFO (coreZkRegister-9130-thread-1-processing-n:127.0.0.1:58617__%2Fqu x:collection1 c:control_collection) [n:127.0.0.1:58617__%2Fqu c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2563671 INFO (coreZkRegister-9130-thread-1-processing-n:127.0.0.1:58617__%2Fqu x:collection1 c:control_collection) [n:127.0.0.1:58617__%2Fqu c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:58617/_/qu/collection1/
[junit4] 2> 2563671 INFO (coreZkRegister-9130-thread-1-processing-n:127.0.0.1:58617__%2Fqu x:collection1 c:control_collection) [n:127.0.0.1:58617__%2Fqu c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 2563671 INFO (coreZkRegister-9130-thread-1-processing-n:127.0.0.1:58617__%2Fqu x:collection1 c:control_collection) [n:127.0.0.1:58617__%2Fqu c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:58617/_/qu/collection1/ has no replicas
[junit4] 2> 2563686 INFO (coreZkRegister-9130-thread-1-processing-n:127.0.0.1:58617__%2Fqu x:collection1 c:control_collection) [n:127.0.0.1:58617__%2Fqu c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:58617/_/qu/collection1/ shard1
[junit4] 2> 2563838 INFO (coreZkRegister-9130-thread-1-processing-n:127.0.0.1:58617__%2Fqu x:collection1 c:control_collection) [n:127.0.0.1:58617__%2Fqu c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2564424 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2564425 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:50432/solr ready
[junit4] 2> 2564425 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 2564425 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
[junit4] 2> 2564573 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_E20191D1B2826B5B-001/shard-1-001/cores/collection1
[junit4] 2> 2564574 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_E20191D1B2826B5B-001/shard-1-001
[junit4] 2> 2564574 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 2564575 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@638f5e18{/_/qu,null,AVAILABLE}
[junit4] 2> 2564576 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3d09f2f9{HTTP/1.1,[http/1.1]}{127.0.0.1:40385}
[junit4] 2> 2564576 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.e.j.s.Server Started @2568397ms
[junit4] 2> 2564576 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_E20191D1B2826B5B-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/_/qu, hostPort=40385, coreRootDirectory=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_E20191D1B2826B5B-001/shard-1-001/cores}
[junit4] 2> 2564577 ERROR (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2564577 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 2564577 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2564577 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2564577 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-13T04:15:33.755Z
[junit4] 2> 2564582 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2564582 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_E20191D1B2826B5B-001/shard-1-001/solr.xml
[junit4] 2> 2564588 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:50432/solr
[junit4] 2> 2564605 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [n:127.0.0.1:40385__%2Fqu ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2564609 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [n:127.0.0.1:40385__%2Fqu ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2564613 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [n:127.0.0.1:40385__%2Fqu ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40385__%2Fqu
[junit4] 2> 2564616 INFO (zkCallback-3849-thread-2-processing-n:127.0.0.1:58617__%2Fqu) [n:127.0.0.1:58617__%2Fqu ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2564616 INFO (zkCallback-3853-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2564618 INFO (zkCallback-3859-thread-1-processing-n:127.0.0.1:40385__%2Fqu) [n:127.0.0.1:40385__%2Fqu ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2564748 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [n:127.0.0.1:40385__%2Fqu ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_E20191D1B2826B5B-001/shard-1-001/cores
[junit4] 2> 2564748 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [n:127.0.0.1:40385__%2Fqu ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 2564752 INFO (OverseerStateUpdate-97449366792896516-127.0.0.1:58617__%2Fqu-n_0000000000) [n:127.0.0.1:58617__%2Fqu ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 2564856 INFO (zkCallback-3859-thread-1-processing-n:127.0.0.1:40385__%2Fqu) [n:127.0.0.1:40385__%2Fqu ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2565764 INFO (coreLoadExecutor-9148-thread-1-processing-n:127.0.0.1:40385__%2Fqu) [n:127.0.0.1:40385__%2Fqu c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2565784 INFO (coreLoadExecutor-9148-thread-1-processing-n:127.0.0.1:40385__%2Fqu) [n:127.0.0.1:40385__%2Fqu c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 2565868 WARN (coreLoadExecutor-9148-thread-1-processing-n:127.0.0.1:40385__%2Fqu) [n:127.0.0.1:40385__%2Fqu c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 2565870 INFO (coreLoadExecutor-9148-thread-1-processing-n:127.0.0.1:40385__%2Fqu) [n:127.0.0.1:40385__%2Fqu c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 2565887 INFO (coreLoadExecutor-9148-thread-1-processing-n:127.0.0.1:40385__%2Fqu) [n:127.0.0.1:40385__%2Fqu c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 2565887 INFO (coreLoadExecutor-9148-thread-1-processing-n:127.0.0.1:40385__%2Fqu) [n:127.0.0.1:40385__%2Fqu c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_E20191D1B2826B5B-001/shard-1-001/cores/collection1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_E20191D1B2826B5B-001/shard-1-001/cores/collection1/data/]
[junit4] 2> 2565887 INFO (coreLoadExecutor-9148-thread-1-processing-n:127.0.0.1:40385__%2Fqu) [n:127.0.0.1:40385__%2Fqu c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@24d53b22
[junit4] 2> 2565889 INFO (coreLoadExecutor-9148-thread-1-processing-n:127.0.0.1:40385__%2Fqu) [n:127.0.0.1:40385__%2Fqu c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=7, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 2565898 WARN (coreLoadExecutor-9148-thread-1-processing-n:127.0.0.1:40385__%2Fqu) [n:127.0.0.1:40385__%2Fqu c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 2565944 INFO (coreLoadExecutor-9148-thread-1-processing-n:127.0.0.1:40385__%2Fqu) [n:127.0.0.1:40385__%2Fqu c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2565944 INFO (coreLoadExecutor-9148-thread-1-processing-n:127.0.0.1:40385__%2Fqu) [n:127.0.0.1:40385__%2Fqu c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=1000 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2565944 INFO (coreLoadExecutor-9148-thread-1-processing-n:127.0.0.1:40385__%2Fqu) [n:127.0.0.1:40385__%2Fqu c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2565944 INFO (coreLoadExecutor-9148-thread-1-processing-n:127.0.0.1:40385__%2Fqu) [n:127.0.0.1:40385__%2Fqu c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2565945 INFO (coreLoadExecutor-9148-thread-1-processing-n:127.0.0.1:40385__%2Fqu) [n:127.0.0.1:40385__%2Fqu c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=19, maxMergeAtOnceExplicit=29, maxMergedSegmentMB=93.8798828125, floorSegmentMB=0.9091796875, forceMergeDeletesPctAllowed=2.908175163128498, segmentsPerTier=13.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7895201225707248
[junit4] 2> 2565946 INFO (coreLoadExecutor-9148-thread-1-processing-n:127.0.0.1:40385__%2Fqu) [n:127.0.0.1:40385__%2Fqu c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@23c0069e[collection1] main]
[junit4] 2> 2565947 INFO (coreLoadExecutor-9148-thread-1-processing-n:127.0.0.1:40385__%2Fqu) [n:127.0.0.1:40385__%2Fqu c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2565948 INFO (coreLoadExecutor-9148-thread-1-processing-n:127.0.0.1:40385__%2Fqu) [n:127.0.0.1:40385__%2Fqu c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2565948 INFO (coreLoadExecutor-9148-thread-1-processing-n:127.0.0.1:40385__%2Fqu) [n:127.0.0.1:40385__%2Fqu c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2565949 INFO (searcherExecutor-9149-thread-1-processing-n:127.0.0.1:40385__%2Fqu x:collection1 c:collection1) [n:127.0.0.1:40385__%2Fqu c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@23c0069e[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2565949 INFO (coreLoadExecutor-9148-thread-1-processing-n:127.0.0.1:40385__%2Fqu) [n:127.0.0.1:40385__%2Fqu c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559189871790129152
[junit4] 2> 2565958 INFO (coreZkRegister-9143-thread-1-processing-n:127.0.0.1:40385__%2Fqu x:collection1 c:collection1) [n:127.0.0.1:40385__%2Fqu c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2565958 INFO (coreZkRegister-9143-thread-1-processing-n:127.0.0.1:40385__%2Fqu x:collection1 c:collection1) [n:127.0.0.1:40385__%2Fqu c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2565959 INFO (coreZkRegister-9143-thread-1-processing-n:127.0.0.1:40385__%2Fqu x:collection1 c:collection1) [n:127.0.0.1:40385__%2Fqu c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:40385/_/qu/collection1/
[junit4] 2> 2565959 INFO (coreZkRegister-9143-thread-1-processing-n:127.0.0.1:40385__%2Fqu x:collection1 c:collection1) [n:127.0.0.1:40385__%2Fqu c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 2565959 INFO (coreZkRegister-9143-thread-1-processing-n:127.0.0.1:40385__%2Fqu x:collection1 c:collection1) [n:127.0.0.1:40385__%2Fqu c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:40385/_/qu/collection1/ has no replicas
[junit4] 2> 2565966 INFO (coreZkRegister-9143-thread-1-processing-n:127.0.0.1:40385__%2Fqu x:collection1 c:collection1) [n:127.0.0.1:40385__%2Fqu c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:40385/_/qu/collection1/ shard1
[junit4] 2> 2566070 INFO (zkCallback-3859-thread-1-processing-n:127.0.0.1:40385__%2Fqu) [n:127.0.0.1:40385__%2Fqu ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2566074 INFO (coreZkRegister-9143-thread-1-processing-n:127.0.0.1:40385__%2Fqu x:collection1 c:collection1) [n:127.0.0.1:40385__%2Fqu c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2566184 INFO (zkCallback-3859-thread-1-processing-n:127.0.0.1:40385__%2Fqu) [n:127.0.0.1:40385__%2Fqu ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2566323 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_E20191D1B2826B5B-001/shard-2-001/cores/collection1
[junit4] 2> 2566324 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_E20191D1B2826B5B-001/shard-2-001
[junit4] 2> 2566324 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 2566326 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@130fdaae{/_/qu,null,AVAILABLE}
[junit4] 2> 2566326 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3129b146{HTTP/1.1,[http/1.1]}{127.0.0.1:37632}
[junit4] 2> 2566329 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.e.j.s.Server Started @2570150ms
[junit4] 2> 2566336 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_E20191D1B2826B5B-001/tempDir-001/jetty2, solrconfig=solrconfig.xml, hostContext=/_/qu, hostPort=37632, coreRootDirectory=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_E20191D1B2826B5B-001/shard-2-001/cores}
[junit4] 2> 2566344 ERROR (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2566344 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 2566345 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2566345 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2566345 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-13T04:15:35.523Z
[junit4] 2> 2566367 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2566367 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_E20191D1B2826B5B-001/shard-2-001/solr.xml
[junit4] 2> 2566375 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:50432/solr
[junit4] 2> 2566530 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [n:127.0.0.1:37632__%2Fqu ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2566534 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [n:127.0.0.1:37632__%2Fqu ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2566539 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [n:127.0.0.1:37632__%2Fqu ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37632__%2Fqu
[junit4] 2> 2566545 INFO (zkCallback-3859-thread-1-processing-n:127.0.0.1:40385__%2Fqu) [n:127.0.0.1:40385__%2Fqu ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2566545 INFO (zkCallback-3849-thread-4-processing-n:127.0.0.1:58617__%2Fqu) [n:127.0.0.1:58617__%2Fqu ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2566545 INFO (zkCallback-3853-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2566581 INFO (zkCallback-3865-thread-2-processing-n:127.0.0.1:37632__%2Fqu) [n:127.0.0.1:37632__%2Fqu ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2566647 INFO (zkCallback-3859-thread-1-processing-n:127.0.0.1:40385__%2Fqu) [n:127.0.0.1:40385__%2Fqu ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2566664 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [n:127.0.0.1:37632__%2Fqu ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_E20191D1B2826B5B-001/shard-2-001/cores
[junit4] 2> 2566664 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [n:127.0.0.1:37632__%2Fqu ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 2566668 INFO (OverseerStateUpdate-97449366792896516-127.0.0.1:58617__%2Fqu-n_0000000000) [n:127.0.0.1:58617__%2Fqu ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 2566772 INFO (zkCallback-3859-thread-1-processing-n:127.0.0.1:40385__%2Fqu) [n:127.0.0.1:40385__%2Fqu ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2566772 INFO (zkCallback-3865-thread-2-processing-n:127.0.0.1:37632__%2Fqu) [n:127.0.0.1:37632__%2Fqu ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2567686 INFO (coreLoadExecutor-9159-thread-1-processing-n:127.0.0.1:37632__%2Fqu) [n:127.0.0.1:37632__%2Fqu c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2567698 INFO (coreLoadExecutor-9159-thread-1-processing-n:127.0.0.1:37632__%2Fqu) [n:127.0.0.1:37632__%2Fqu c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 2567784 WARN (coreLoadExecutor-9159-thread-1-processing-n:127.0.0.1:37632__%2Fqu) [n:127.0.0.1:37632__%2Fqu c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 2567786 INFO (coreLoadExecutor-9159-thread-1-processing-n:127.0.0.1:37632__%2Fqu) [n:127.0.0.1:37632__%2Fqu c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 2567804 INFO (coreLoadExecutor-9159-thread-1-processing-n:127.0.0.1:37632__%2Fqu) [n:127.0.0.1:37632__%2Fqu c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 2567805 INFO (coreLoadExecutor-9159-thread-1-processing-n:127.0.0.1:37632__%2Fqu) [n:127.0.0.1:37632__%2Fqu c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_E20191D1B2826B5B-001/shard-2-001/cores/collection1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_E20191D1B2826B5B-001/shard-2-001/cores/collection1/data/]
[junit4] 2> 2567805 INFO (coreLoadExecutor-9159-thread-1-processing-n:127.0.0.1:37632__%2Fqu) [n:127.0.0.1:37632__%2Fqu c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@24d53b22
[junit4] 2> 2567809 INFO (coreLoadExecutor-9159-thread-1-processing-n:127.0.0.1:37632__%2Fqu) [n:127.0.0.1:37632__%2Fqu c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=7, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 2567820 WARN (coreLoadExecutor-9159-thread-1-processing-n:127.0.0.1:37632__%2Fqu) [n:127.0.0.1:37632__%2Fqu c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 2567865 INFO (coreLoadExecutor-9159-thread-1-processing-n:127.0.0.1:37632__%2Fqu) [n:127.0.0.1:37632__%2Fqu c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2567865 INFO (coreLoadExecutor-9159-thread-1-processing-n:127.0.0.1:37632__%2Fqu) [n:127.0.0.1:37632__%2Fqu c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=1000 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2567866 INFO (coreLoadExecutor-9159-thread-1-processing-n:127.0.0.1:37632__%2Fqu) [n:127.0.0.1:37632__%2Fqu c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2567866 INFO (coreLoadExecutor-9159-thread-1-processing-n:127.0.0.1:37632__%2Fqu) [n:127.0.0.1:37632__%2Fqu c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2567867 INFO (coreLoadExecutor-9159-thread-1-processing-n:127.0.0.1:37632__%2Fqu) [n:127.0.0.1:37632__%2Fqu c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=19, maxMergeAtOnceExplicit=29, maxMergedSegmentMB=93.8798828125, floorSegmentMB=0.9091796875, forceMergeDeletesPctAllowed=2.908175163128498, segmentsPerTier=13.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7895201225707248
[junit4] 2> 2567867 INFO (coreLoadExecutor-9159-thread-1-processing-n:127.0.0.1:37632__%2Fqu) [n:127.0.0.1:37632__%2Fqu c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@25e105ea[collection1] main]
[junit4] 2> 2567869 INFO (coreLoadExecutor-9159-thread-1-processing-n:127.0.0.1:37632__%2Fqu) [n:127.0.0.1:37632__%2Fqu c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2567870 INFO (coreLoadExecutor-9159-thread-1-processing-n:127.0.0.1:37632__%2Fqu) [n:127.0.0.1:37632__%2Fqu c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2567870 INFO (coreLoadExecutor-9159-thread-1-processing-n:127.0.0.1:37632__%2Fqu) [n:127.0.0.1:37632__%2Fqu c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2567872 INFO (searcherExecutor-9160-thread-1-processing-n:127.0.0.1:37632__%2Fqu x:collection1 c:collection1) [n:127.0.0.1:37632__%2Fqu c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@25e105ea[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2567872 INFO (coreLoadExecutor-9159-thread-1-processing-n:127.0.0.1:37632__%2Fqu) [n:127.0.0.1:37632__%2Fqu c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559189873806540800
[junit4] 2> 2567878 INFO (coreZkRegister-9154-thread-1-processing-n:127.0.0.1:37632__%2Fqu x:collection1 c:collection1) [n:127.0.0.1:37632__%2Fqu c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ZkController Core needs to recover:collection1
[junit4] 2> 2567879 INFO (updateExecutor-3862-thread-1-processing-n:127.0.0.1:37632__%2Fqu x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:37632__%2Fqu c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.DefaultSolrCoreState Running recovery
[junit4] 2> 2567880 INFO (recoveryExecutor-3863-thread-1-processing-n:127.0.0.1:37632__%2Fqu x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:37632__%2Fqu c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
[junit4] 2> 2567880 INFO (recoveryExecutor-3863-thread-1-processing-n:127.0.0.1:37632__%2Fqu x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:37632__%2Fqu c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
[junit4] 2> 2567880 INFO (recoveryExecutor-3863-thread-1-processing-n:127.0.0.1:37632__%2Fqu x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:37632__%2Fqu c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1]
[junit4] 2> 2567880 INFO (recoveryExecutor-3863-thread-1-processing-n:127.0.0.1:37632__%2Fqu x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:37632__%2Fqu c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 2567880 INFO (recoveryExecutor-3863-thread-1-processing-n:127.0.0.1:37632__%2Fqu x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:37632__%2Fqu c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Publishing state of core [collection1] as recovering, leader is [http://127.0.0.1:40385/_/qu/collection1/] and I am [http://127.0.0.1:37632/_/qu/collection1/]
[junit4] 2> 2567884 INFO (recoveryExecutor-3863-thread-1-processing-n:127.0.0.1:37632__%2Fqu x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:37632__%2Fqu c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:40385/_/qu]; [WaitForState: action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1:37632__%252Fqu&coreNodeName=core_node2&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
[junit4] 2> 2567887 INFO (qtp1902545071-24128) [n:127.0.0.1:40385__%2Fqu ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node2, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
[junit4] 2> 2567888 INFO (qtp1902545071-24128) [n:127.0.0.1:40385__%2Fqu ] o.a.s.h.a.PrepRecoveryOp Will wait a max of 183 seconds to see collection1 (shard1 of collection1) have state: recovering
[junit4] 2> 2567888 INFO (qtp1902545071-24128) [n:127.0.0.1:40385__%2Fqu ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:37632__%2Fqu, coreNodeName=core_node2, onlyIfActiveCheckResult=false, nodeProps: core_node2:{"core":"collection1","base_url":"http://127.0.0.1:37632/_/qu","node_name":"127.0.0.1:37632__%2Fqu","state":"down"}
[junit4] 2> 2567985 INFO (zkCallback-3859-thread-1-processing-n:127.0.0.1:40385__%2Fqu) [n:127.0.0.1:40385__%2Fqu ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2567985 INFO (zkCallback-3865-thread-2-processing-n:127.0.0.1:37632__%2Fqu) [n:127.0.0.1:37632__%2Fqu ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2568299 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_E20191D1B2826B5B-001/shard-3-001/cores/collection1
[junit4] 2> 2568300 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_E20191D1B2826B5B-001/shard-3-001
[junit4] 2> 2568300 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 2568302 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1a281e73{/_/qu,null,AVAILABLE}
[junit4] 2> 2568302 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@529b6eb7{HTTP/1.1,[http/1.1]}{127.0.0.1:47613}
[junit4] 2> 2568302 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.e.j.s.Server Started @2572125ms
[junit4] 2> 2568302 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_E20191D1B2826B5B-001/tempDir-001/jetty3, solrconfig=solrconfig.xml, hostContext=/_/qu, hostPort=47613, coreRootDirectory=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_E20191D1B2826B5B-001/shard-3-001/cores}
[junit4] 2> 2568303 ERROR (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2568304 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 2568304 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2568304 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2568304 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-13T04:15:37.482Z
[junit4] 2> 2568308 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2568308 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_E20191D1B2826B5B-001/shard-3-001/solr.xml
[junit4] 2> 2568314 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:50432/solr
[junit4] 2> 2568332 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [n:127.0.0.1:47613__%2Fqu ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 2568335 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [n:127.0.0.1:47613__%2Fqu ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2568343 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [n:127.0.0.1:47613__%2Fqu ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47613__%2Fqu
[junit4] 2> 2568348 INFO (zkCallback-3859-thread-1-processing-n:127.0.0.1:40385__%2Fqu) [n:127.0.0.1:40385__%2Fqu ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2568348 INFO (zkCallback-3849-thread-4-processing-n:127.0.0.1:58617__%2Fqu) [n:127.0.0.1:58617__%2Fqu ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2568350 INFO (zkCallback-3853-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2568350 INFO (zkCallback-3865-thread-2-processing-n:127.0.0.1:37632__%2Fqu) [n:127.0.0.1:37632__%2Fqu ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2568352 INFO (zkCallback-3872-thread-2-processing-n:127.0.0.1:47613__%2Fqu) [n:127.0.0.1:47613__%2Fqu ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2568452 INFO (zkCallback-3859-thread-1-processing-n:127.0.0.1:40385__%2Fqu) [n:127.0.0.1:40385__%2Fqu ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2568452 INFO (zkCallback-3865-thread-2-processing-n:127.0.0.1:37632__%2Fqu) [n:127.0.0.1:37632__%2Fqu ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2568462 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [n:127.0.0.1:47613__%2Fqu ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_E20191D1B2826B5B-001/shard-3-001/cores
[junit4] 2> 2568462 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [n:127.0.0.1:47613__%2Fqu ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 2568468 INFO (OverseerStateUpdate-97449366792896516-127.0.0.1:58617__%2Fqu-n_0000000000) [n:127.0.0.1:58617__%2Fqu ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 2568572 INFO (zkCallback-3872-thread-1-processing-n:127.0.0.1:47613__%2Fqu) [n:127.0.0.1:47613__%2Fqu ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2568574 INFO (zkCallback-3865-thread-2-processing-n:127.0.0.1:37632__%2Fqu) [n:127.0.0.1:37632__%2Fqu ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2568585 INFO (zkCallback-3859-thread-1-processing-n:127.0.0.1:40385__%2Fqu) [n:127.0.0.1:40385__%2Fqu ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2568890 INFO (qtp1902545071-24128) [n:127.0.0.1:40385__%2Fqu ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:37632__%2Fqu, coreNodeName=core_node2, onlyIfActiveCheckResult=false, nodeProps: core_node2:{"core":"collection1","base_url":"http://127.0.0.1:37632/_/qu","node_name":"127.0.0.1:37632__%2Fqu","state":"recovering"}
[junit4] 2> 2568890 INFO (qtp1902545071-24128) [n:127.0.0.1:40385__%2Fqu ] o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node2, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4] 2> 2568890 INFO (qtp1902545071-24128) [n:127.0.0.1:40385__%2Fqu ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:37632__%252Fqu&onlyIfLeaderActive=true&core=collection1&coreNodeName=core_node2&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=1003
[junit4] 2> 2569490 INFO (coreLoadExecutor-9170-thread-1-processing-n:127.0.0.1:47613__%2Fqu) [n:127.0.0.1:47613__%2Fqu c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2569504 INFO (coreLoadExecutor-9170-thread-1-processing-n:127.0.0.1:47613__%2Fqu) [n:127.0.0.1:47613__%2Fqu c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 2569636 WARN (coreLoadExecutor-9170-thread-1-processing-n:127.0.0.1:47613__%2Fqu) [n:127.0.0.1:47613__%2Fqu c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 2569638 INFO (coreLoadExecutor-9170-thread-1-processing-n:127.0.0.1:47613__%2Fqu) [n:127.0.0.1:47613__%2Fqu c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 2569680 INFO (coreLoadExecutor-9170-thread-1-processing-n:127.0.0.1:47613__%2Fqu) [n:127.0.0.1:47613__%2Fqu c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 2569681 INFO (coreLoadExecutor-9170-thread-1-processing-n:127.0.0.1:47613__%2Fqu) [n:127.0.0.1:47613__%2Fqu c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_E20191D1B2826B5B-001/shard-3-001/cores/collection1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_E20191D1B2826B5B-001/shard-3-001/cores/collection1/data/]
[junit4] 2> 2569681 INFO (coreLoadExecutor-9170-thread-1-processing-n:127.0.0.1:47613__%2Fqu) [n:127.0.0.1:47613__%2Fqu c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@24d53b22
[junit4] 2> 2569688 INFO (coreLoadExecutor-9170-thread-1-processing-n:127.0.0.1:47613__%2Fqu) [n:127.0.0.1:47613__%2Fqu c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=7, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 2569712 WARN (coreLoadExecutor-9170-thread-1-processing-n:127.0.0.1:47613__%2Fqu) [n:127.0.0.1:47613__%2Fqu c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 2569816 INFO (coreLoadExecutor-9170-thread-1-processing-n:127.0.0.1:47613__%2Fqu) [n:127.0.0.1:47613__%2Fqu c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2569816 INFO (coreLoadExecutor-9170-thread-1-processing-n:127.0.0.1:47613__%2Fqu) [n:127.0.0.1:47613__%2Fqu c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=1000 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2569819 INFO (coreLoadExecutor-9170-thread-1-processing-n:127.0.0.1:47613__%2Fqu) [n:127.0.0.1:47613__%2Fqu c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2569819 INFO (coreLoadExecutor-9170-thread-1-processing-n:127.0.0.1:47613__%2Fqu) [n:127.0.0.1:47613__%2Fqu c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2569821 INFO (coreLoadExecutor-9170-thread-1-processing-n:127.0.0.1:47613__%2Fqu) [n:127.0.0.1:47613__%2Fqu c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=19, maxMergeAtOnceExplicit=29, maxMergedSegmentMB=93.8798828125, floorSegmentMB=0.9091796875, forceMergeDeletesPctAllowed=2.908175163128498, segmentsPerTier=13.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7895201225707248
[junit4] 2> 2569824 INFO (coreLoadExecutor-9170-thread-1-processing-n:127.0.0.1:47613__%2Fqu) [n:127.0.0.1:47613__%2Fqu c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2d28835f[collection1] main]
[junit4] 2> 2569828 INFO (coreLoadExecutor-9170-thread-1-processing-n:127.0.0.1:47613__%2Fqu) [n:127.0.0.1:47613__%2Fqu c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2569829 INFO (coreLoadExecutor-9170-thread-1-processing-n:127.0.0.1:47613__%2Fqu) [n:127.0.0.1:47613__%2Fqu c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2569830 INFO (coreLoadExecutor-9170-thread-1-processing-n:127.0.0.1:47613__%2Fqu) [n:127.0.0.1:47613__%2Fqu c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2569832 INFO (searcherExecutor-9171-thread-1-processing-n:127.0.0.1:47613__%2Fqu x:collection1 c:collection1) [n:127.0.0.1:47613__%2Fqu c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2d28835f[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2569834 INFO (coreLoadExecutor-9170-thread-1-processing-n:127.0.0.1:47613__%2Fqu) [n:127.0.0.1:47613__%2Fqu c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559189875863846912
[junit4] 2> 2569846 INFO (coreZkRegister-9165-thread-1-processing-n:127.0.0.1:47613__%2Fqu x:collection1 c:collection1) [n:127.0.0.1:47613__%2Fqu c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.ZkController Core needs to recover:collection1
[junit4] 2> 2569849 INFO (updateExecutor-3869-thread-1-processing-n:127.0.0.1:47613__%2Fqu x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:47613__%2Fqu c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.DefaultSolrCoreState Running recovery
[junit4] 2> 2569850 INFO (recoveryExecutor-3870-thread-1-processing-n:127.0.0.1:47613__%2Fqu x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:47613__%2Fqu c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
[junit4] 2> 2569851 INFO (recoveryExecutor-3870-thread-1-processing-n:127.0.0.1:47613__%2Fqu x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:47613__%2Fqu c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
[junit4] 2> 2569851 INFO (recoveryExecutor-3870-thread-1-processing-n:127.0.0.1:47613__%2Fqu x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:47613__%2Fqu c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1]
[junit4] 2> 2569851 INFO (recoveryExecutor-3870-thread-1-processing-n:127.0.0.1:47613__%2Fqu x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:47613__%2Fqu c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 2569851 INFO (recoveryExecutor-3870-thread-1-processing-n:127.0.0.1:47613__%2Fqu x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:47613__%2Fqu c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Publishing state of core [collection1] as recovering, leader is [http://127.0.0.1:40385/_/qu/collection1/] and I am [http://127.0.0.1:47613/_/qu/collection1/]
[junit4] 2> 2569861 INFO (recoveryExecutor-3870-thread-1-processing-n:127.0.0.1:47613__%2Fqu x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:47613__%2Fqu c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:40385/_/qu]; [WaitForState: action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1:47613__%252Fqu&coreNodeName=core_node3&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
[junit4] 2> 2569863 INFO (qtp1902545071-24132) [n:127.0.0.1:40385__%2Fqu ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
[junit4] 2> 2569867 INFO (qtp1902545071-24132) [n:127.0.0.1:40385__%2Fqu ] o.a.s.h.a.PrepRecoveryOp Will wait a max of 183 seconds to see collection1 (shard1 of collection1) have state: recovering
[junit4] 2> 2569867 INFO (qtp1902545071-24132) [n:127.0.0.1:40385__%2Fqu ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:47613__%2Fqu, coreNodeName=core_node3, onlyIfActiveCheckResult=false, nodeProps: core_node3:{"core":"collection1","base_url":"http://127.0.0.1:47613/_/qu","node_name":"127.0.0.1:47613__%2Fqu","state":"down"}
[junit4] 2> 2569872 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.SolrTestCaseJ4 ###Starting test
[junit4] 2> 2569872 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Wait for recoveries to finish - wait 30 for each attempt
[junit4] 2> 2569873 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):30
[junit4] 2> 2569971 INFO (zkCallback-3872-thread-1-processing-n:127.0.0.1:47613__%2Fqu) [n:127.0.0.1:47613__%2Fqu ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2569971 INFO (zkCallback-3859-thread-1-processing-n:127.0.0.1:40385__%2Fqu) [n:127.0.0.1:40385__%2Fqu ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2569971 INFO (zkCallback-3865-thread-2-processing-n:127.0.0.1:37632__%2Fqu) [n:127.0.0.1:37632__%2Fqu ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2570873 INFO (qtp1902545071-24132) [n:127.0.0.1:40385__%2Fqu ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:47613__%2Fqu, coreNodeName=core_node3, onlyIfActiveCheckResult=false, nodeProps: core_node3:{"core":"collection1","base_url":"http://127.0.0.1:47613/_/qu","node_name":"127.0.0.1:47613__%2Fqu","state":"recovering"}
[junit4] 2> 2570873 INFO (qtp1902545071-24132) [n:127.0.0.1:40385__%2Fqu ] o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4] 2> 2570874 INFO (qtp1902545071-24132) [n:127.0.0.1:40385__%2Fqu ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:47613__%252Fqu&onlyIfLeaderActive=true&core=collection1&coreNodeName=core_node3&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=1011
[junit4] 2> 2575891 INFO (recoveryExecutor-3863-thread-1-processing-n:127.0.0.1:37632__%2Fqu x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:37632__%2Fqu c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Attempting to PeerSync from [http://127.0.0.1:40385/_/qu/collection1/] - recoveringAfterStartup=[true]
[junit4] 2> 2575891 INFO (recoveryExecutor-3863-thread-1-processing-n:127.0.0.1:37632__%2Fqu x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:37632__%2Fqu c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.PeerSync PeerSync: core=collection1 url=http://127.0.0.1:37632/_/qu START replicas=[http://127.0.0.1:40385/_/qu/collection1/] nUpdates=1000
[junit4] 2> 2575897 INFO (qtp1902545071-24126) [n:127.0.0.1:40385__%2Fqu c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.IndexFingerprint IndexFingerprint millis:0.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
[junit4] 2> 2575897 INFO (qtp1902545071-24126) [n:127.0.0.1:40385__%2Fqu c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.S.Request [collection1] webapp=/_/qu path=/get params={distrib=false&qt=/get&getFingerprint=9223372036854775807&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2575898 INFO (recoveryExecutor-3863-thread-1-processing-n:127.0.0.1:37632__%2Fqu x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:37632__%2Fqu c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.IndexFingerprint IndexFingerprint millis:0.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
[junit4] 2> 2575898 INFO (recoveryExecutor-3863-thread-1-processing-n:127.0.0.1:37632__%2Fqu x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:37632__%2Fqu c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.PeerSync We are already in sync. No need to do a PeerSync
[junit4] 2> 2575898 INFO (recoveryExecutor-3863-thread-1-processing-n:127.0.0.1:37632__%2Fqu x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:37632__%2Fqu c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2575898 INFO (recoveryExecutor-3863-thread-1-processing-n:127.0.0.1:37632__%2Fqu x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:37632__%2Fqu c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 2575898 INFO (recoveryExecutor-3863-thread-1-processing-n:127.0.0.1:37632__%2Fqu x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:37632__%2Fqu c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2575899 INFO (recoveryExecutor-3863-thread-1-processing-n:127.0.0.1:37632__%2Fqu x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:37632__%2Fqu c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy PeerSync stage of recovery was successful.
[junit4] 2> 2575899 INFO (recoveryExecutor-3863-thread-1-processing-n:127.0.0.1:37632__%2Fqu x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:37632__%2Fqu c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Replaying updates buffered during PeerSync.
[junit4] 2> 2575899 INFO (recoveryExecutor-3863-thread-1-processing-n:127.0.0.1:37632__%2Fqu x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:37632__%2Fqu c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy No replay needed.
[junit4] 2> 2575899 INFO (recoveryExecutor-3863-thread-1-processing-n:127.0.0.1:37632__%2Fqu x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:37632__%2Fqu c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Registering as Active after recovery.
[junit4] 2> 2575906 INFO (zkCallback-3859-thread-1-processing-n:127.0.0.1:40385__%2Fqu) [n:127.0.0.1:40385__%2Fqu ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2575908 INFO (zkCallback-3865-thread-2-processing-n:127.0.0.1:37632__%2Fqu) [n:127.0.0.1:37632__%2Fqu ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2575909 INFO (zkCallback-3872-thread-1-processing-n:127.0.0.1:47613__%2Fqu) [n:127.0.0.1:47613__%2Fqu ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2577874 INFO (recoveryExecutor-3870-thread-1-processing-n:127.0.0.1:47613__%2Fqu x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:47613__%2Fqu c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Attempting to PeerSync from [http://127.0.0.1:40385/_/qu/collection1/] - recoveringAfterStartup=[true]
[junit4] 2> 2577875 INFO (recoveryExecutor-3870-thread-1-processing-n:127.0.0.1:47613__%2Fqu x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:47613__%2Fqu c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.PeerSync PeerSync: core=collection1 url=http://127.0.0.1:47613/_/qu START replicas=[http://127.0.0.1:40385/_/qu/collection1/] nUpdates=1000
[junit4] 2> 2577876 INFO (qtp1902545071-24131) [n:127.0.0.1:40385__%2Fqu c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.IndexFingerprint IndexFingerprint millis:0.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
[junit4] 2> 2577876 INFO (qtp1902545071-24131) [n:127.0.0.1:40385__%2Fqu c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.S.Request [collection1] webapp=/_/qu path=/get params={distrib=false&qt=/get&getFingerprint=9223372036854775807&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2577878 INFO (recoveryExecutor-3870-thread-1-processing-n:127.0.0.1:47613__%2Fqu x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:47613__%2Fqu c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.IndexFingerprint IndexFingerprint millis:0.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
[junit4] 2> 2577878 INFO (recoveryExecutor-3870-thread-1-processing-n:127.0.0.1:47613__%2Fqu x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:47613__%2Fqu c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.PeerSync We are already in sync. No need to do a PeerSync
[junit4] 2> 2577878 INFO (recoveryExecutor-3870-thread-1-processing-n:127.0.0.1:47613__%2Fqu x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:47613__%2Fqu c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2577878 INFO (recoveryExecutor-3870-thread-1-processing-n:127.0.0.1:47613__%2Fqu x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:47613__%2Fqu c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 2577878 INFO (recoveryExecutor-3870-thread-1-processing-n:127.0.0.1:47613__%2Fqu x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:47613__%2Fqu c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2577878 INFO (recoveryExecutor-3870-thread-1-processing-n:127.0.0.1:47613__%2Fqu x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:47613__%2Fqu c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy PeerSync stage of recovery was successful.
[junit4] 2> 2577878 INFO (recoveryExecutor-3870-thread-1-processing-n:127.0.0.1:47613__%2Fqu x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:47613__%2Fqu c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Replaying updates buffered during PeerSync.
[junit4] 2> 2577878 INFO (recoveryExecutor-3870-thread-1-processing-n:127.0.0.1:47613__%2Fqu x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:47613__%2Fqu c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy No replay needed.
[junit4] 2> 2577878 INFO (recoveryExecutor-3870-thread-1-processing-n:127.0.0.1:47613__%2Fqu x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:47613__%2Fqu c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Registering as Active after recovery.
[junit4] 2> 2577882 INFO (zkCallback-3859-thread-1-processing-n:127.0.0.1:40385__%2Fqu) [n:127.0.0.1:40385__%2Fqu ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2577882 INFO (zkCallback-3872-thread-1-processing-n:127.0.0.1:47613__%2Fqu) [n:127.0.0.1:47613__%2Fqu ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2577883 INFO (zkCallback-3865-thread-2-processing-n:127.0.0.1:37632__%2Fqu) [n:127.0.0.1:37632__%2Fqu ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2577901 INFO (TEST-PeerSyncReplicationTest.test-seed#[E20191D1B2826B5B]) [ ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: collection1
[junit4] 2> 2577904 INFO (qtp687993528-24087) [n:127.0.0.1:58617__%2Fqu c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2577904 INFO (qtp687993528-24087) [n:127.0.0.1:58617__%2Fqu c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 2577904 INFO (qtp687993528-24087) [n:127.0.0.1:58617__%2Fqu c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2577904 INFO (qtp687993528-24087) [n:127.0.0.1:58617__%2Fqu c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/_/qu path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 0
[junit4] 2> 2577907 INFO (qtp1902545071-24133) [n:127.0.0.1:40385__%2Fqu c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2577907 INFO (qtp1902545071-24133) [n:127.0.0.1:40385__%2Fqu c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 2577908 INFO (qtp1269447483-24154) [n:127.0.0.1:37632__%2Fqu c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2577908 INFO (qtp1902545071-24133) [n:127.0.0.1:40385__%2Fqu c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2577908 INFO (qtp1269447483-24154) [n:127.0.0.1:37632__%2Fqu c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 2577908 INFO (qtp1902545071-24133) [n:127.0.0.1:40385__%2Fqu c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/_/qu path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:40385/_/qu/collection1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 1
[junit4] 2> 2577908 INFO (qtp1269447483-24154) [n:127.0.0.1:37632__%2Fqu c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2577909 INFO (qtp1269447483-24154) [n:127.0.0.1:37632__%2Fqu c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/_/qu path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:40385/_/qu/collection1/&commit_end_point=tr
[...truncated too long message...]
/solr/configs/conf1
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 6 /solr/clusterstate.json
[junit4] 2> 4 /solr/collections/collection1/state.json
[junit4] 2> 2 /solr/collections/collection1/leader_elect/shard1/election/97449366792896521-core_node1-n_0000000000
[junit4] 2> 2 /solr/overseer_elect/election/97449366792896521-127.0.0.1:40385__%2Fqu-n_0000000001
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 207 /solr/overseer/collection-queue-work
[junit4] 2> 44 /solr/overseer/queue
[junit4] 2> 15 /solr/overseer/queue-work
[junit4] 2> 6 /solr/collections
[junit4] 2> 5 /solr/live_nodes
[junit4] 2>
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=PeerSyncReplicationTest -Dtests.method=test -Dtests.seed=E20191D1B2826B5B -Dtests.slow=true -Dtests.locale=tr -Dtests.timezone=Pacific/Niue -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] FAILURE 213s J1 | PeerSyncReplicationTest.test <<<
[junit4] > Throwable #1: java.lang.AssertionError: timeout waiting to see all nodes active
[junit4] > at __randomizedtesting.SeedInfo.seed([E20191D1B2826B5B:6A55AE0B1C7E06A3]:0)
[junit4] > at org.apache.solr.cloud.PeerSyncReplicationTest.waitTillNodesActive(PeerSyncReplicationTest.java:326)
[junit4] > at org.apache.solr.cloud.PeerSyncReplicationTest.bringUpDeadNodeAndEnsureNoReplication(PeerSyncReplicationTest.java:277)
[junit4] > at org.apache.solr.cloud.PeerSyncReplicationTest.forceNodeFailureAndDoPeerSync(PeerSyncReplicationTest.java:259)
[junit4] > at org.apache.solr.cloud.PeerSyncReplicationTest.test(PeerSyncReplicationTest.java:138)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 2775497 INFO (SUITE-PeerSyncReplicationTest-seed#[E20191D1B2826B5B]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_E20191D1B2826B5B-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {other_tl1=PostingsFormat(name=LuceneVarGapFixedInterval), range_facet_l_dv=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), rnd_s=PostingsFormat(name=LuceneVarGapFixedInterval), multiDefault=BlockTreeOrds(blocksize=128), a_t=BlockTreeOrds(blocksize=128), intDefault=PostingsFormat(name=LuceneVarGapFixedInterval), a_i1=BlockTreeOrds(blocksize=128), id=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), range_facet_i_dv=BlockTreeOrds(blocksize=128), text=PostingsFormat(name=LuceneVarGapDocFreqInterval), range_facet_l=BlockTreeOrds(blocksize=128), timestamp=BlockTreeOrds(blocksize=128)}, docValues:{range_facet_l_dv=DocValuesFormat(name=Direct), _version_=DocValuesFormat(name=Memory), range_facet_i_dv=DocValuesFormat(name=Lucene70), intDvoDefault=DocValuesFormat(name=Direct), timestamp=DocValuesFormat(name=Lucene70)}, maxPointsInLeafNode=254, maxMBSortInHeap=6.987966635983616, sim=RandomSimilarity(queryNorm=false): {}, locale=tr, timezone=Pacific/Niue
[junit4] 2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_121 (64-bit)/cpus=3,threads=1,free=178207440,total=518979584
[junit4] 2> NOTE: All tests run in this JVM: [TestConfigSetsAPIZkFailure, TestFastWriter, SuggesterWFSTTest, RankQueryTest, CoreAdminCreateDiscoverTest, ClusterStateTest, TestCrossCoreJoin, SolrJmxReporterTest, TestManagedStopFilterFactory, TestLMDirichletSimilarityFactory, TestIndexingPerformance, TestConfig, TestExceedMaxTermLength, TestFieldCacheSanityChecker, TestDocSet, TestNonDefinedSimilarityFactory, TestWriterPerf, OverseerModifyCollectionTest, ChangedSchemaMergeTest, SpellingQueryConverterTest, TestSearchPerf, TermVectorComponentDistributedTest, ShardRoutingCustomTest, MultiThreadedOCPTest, BlockJoinFacetRandomTest, TestSearcherReuse, TestNumericTerms32, DateMathParserTest, SolrIndexSplitterTest, MigrateRouteKeyTest, TestSerializedLuceneMatchVersion, UUIDUpdateProcessorFallbackTest, TestCollapseQParserPlugin, CachingDirectoryFactoryTest, SolrGangliaReporterTest, TestSolrCoreProperties, HdfsWriteToMultipleCollectionsTest, LukeRequestHandlerTest, TestGeoJSONResponseWriter, UpdateLogTest, TestSubQueryTransformer, TestSchemaSimilarityResource, NoCacheHeaderTest, TestBackupRepositoryFactory, SpellCheckComponentTest, TestComponentsName, CurrencyFieldOpenExchangeTest, TestDistributedMissingSort, TestPKIAuthenticationPlugin, TestSolrCoreSnapshots, GraphQueryTest, TestMacros, URLClassifyProcessorTest, SaslZkACLProviderTest, TestCloudPseudoReturnFields, TestFieldCacheSortRandom, SpellPossibilityIteratorTest, TestApiFramework, TestRecoveryHdfs, SimpleCollectionCreateDeleteTest, TestFieldCacheWithThreads, BadCopyFieldTest, AddSchemaFieldsUpdateProcessorFactoryTest, SmileWriterTest, ShowFileRequestHandlerTest, TestSolrCoreParser, TestUninvertingReader, TestDownShardTolerantSearch, TestManagedSchema, BlobRepositoryCloudTest, RemoteQueryErrorTest, TestBulkSchemaAPI, TestFieldCache, TestRecovery, TestPHPSerializedResponseWriter, TestReversedWildcardFilterFactory, TestOrdValues, HttpPartitionTest, ConcurrentDeleteAndCreateCollectionTest, WrapperMergePolicyFactoryTest, ClassificationUpdateProcessorTest, TlogReplayBufferedWhileIndexingTest, TestInitQParser, FieldAnalysisRequestHandlerTest, MetricsHandlerTest, CustomCollectionTest, TestFieldSortValues, JsonLoaderTest, LeaderElectionIntegrationTest, BaseCdcrDistributedZkTest, TestAuthenticationFramework, TestRequestForwarding, ZkSolrClientTest, PreAnalyzedFieldTest, TestFiltering, ManagedSchemaRoundRobinCloudTest, BitVectorTest, TestSQLHandler, TestNoOpRegenerator, SolrCloudExampleTest, HLLUtilTest, TestSolrConfigHandlerCloud, TestAtomicUpdateErrorCases, SpellCheckCollatorTest, CSVRequestHandlerTest, DirectUpdateHandlerOptimizeTest, TestImpersonationWithHadoopAuth, DistributedFacetPivotLongTailTest, TestIndexSearcher, SuggesterTest, TestBadConfig, SolrCmdDistributorTest, TestExactStatsCache, DistanceFunctionTest, TestComplexPhraseQParserPlugin, TestDFRSimilarityFactory, DocExpirationUpdateProcessorFactoryTest, CursorMarkTest, FieldMutatingUpdateProcessorTest, TestJsonFacets, TestUnifiedSolrHighlighter, TestOmitPositions, HdfsNNFailoverTest, TestSuggestSpellingConverter, HdfsRecoveryZkTest, ReplicationFactorTest, TestInitParams, TestSolrConfigHandler, UpdateRequestProcessorFactoryTest, TestCoreDiscovery, SuggesterTSTTest, TestSolrIndexConfig, CollectionsAPIAsyncDistributedZkTest, TestTrieFacet, TestLeaderElectionZkExpiry, DistributedIntervalFacetingTest, BasicDistributedZk2Test, LeaderInitiatedRecoveryOnCommitTest, ConjunctionSolrSpellCheckerTest, CdcrRequestHandlerTest, TermVectorComponentTest, TestValueSourceCache, TestHdfsCloudBackupRestore, LeaderFailoverAfterPartitionTest, ZkStateWriterTest, CollectionStateFormat2Test, SolrCoreMetricManagerTest, CdcrReplicationDistributedZkTest, SolrMetricReporterTest, TestDistributedStatsComponentCardinality, WordBreakSolrSpellCheckerTest, DeleteLastCustomShardedReplicaTest, TestRangeQuery, DeleteReplicaTest, TestTestInjection, ActionThrottleTest, TestCloudManagedSchema, UpdateParamsTest, UUIDFieldTest, TestSolr4Spatial, InfixSuggestersTest, TestReloadAndDeleteDocs, AutoCommitTest, TestSolrXml, TestSolrQueryResponse, TestSSLRandomization, DistributedFacetExistsSmallTest, BasicDistributedZkTest, ChaosMonkeySafeLeaderTest, SyncSliceTest, OpenCloseCoreStressTest, OverseerTest, BasicZkTest, FullSolrCloudDistribCmdsTest, ClusterStateUpdateTest, TestRandomFaceting, TestDistributedSearch, TestRealTimeGet, TestStressReorder, TestReload, DistributedTermsComponentTest, TestGroupingSearch, BadIndexSchemaTest, ConvertedLegacyTest, TestSort, TestFunctionQuery, BasicFunctionalityTest, DistributedQueryElevationComponentTest, SolrRequestParserTest, TestFoldingMultitermQuery, DocValuesTest, TestSolrDeletionPolicy1, DebugComponentTest, TestQueryUtils, StandardRequestHandlerTest, PrimitiveFieldTypeTest, RequestHandlersTest, MoreLikeThisHandlerTest, RequiredFieldsTest, IndexSchemaRuntimeFieldTest, RegexBoostProcessorTest, TestJmxIntegration, ReturnFieldsTest, QueryParsingTest, UniqFieldsUpdateProcessorFactoryTest, TestPartialUpdateDeduplication, SearchHandlerTest, HighlighterConfigTest, TestQuerySenderListener, AlternateDirectoryTest, ResponseLogComponentTest, BadComponentTest, TestStressRecovery, TestSolrDeletionPolicy2, MultiTermTest, NumericFieldsTest, MinimalSchemaTest, TestFuzzyAnalyzedSuggestions, ExternalFileFieldSortTest, NotRequiredUniqueKeyTest, TestLuceneMatchVersion, TestCharFilters, TestCodecSupport, SynonymTokenizerTest, EchoParamsTest, TestPerFieldSimilarity, TestLMJelinekMercerSimilarityFactory, TestBM25SimilarityFactory, TestJmxMonitoredMap, TimeZoneUtilsTest, ResourceLoaderTest, ScriptEngineTest, OpenExchangeRatesOrgProviderTest, ChaosMonkeyNothingIsSafeTest, TestCursorMarkWithoutUniqueKey, TestHighlightDedupGrouping, TestSimpleTrackingShardHandler, CollectionTooManyReplicasTest, ConnectionManagerTest, CreateCollectionCleanupTest, DeleteShardTest, DistribDocExpirationUpdateProcessorTest, LeaderElectionContextKeyTest, LeaderFailureAfterFreshStartTest, LeaderInitiatedRecoveryOnShardRestartTest, MissingSegmentRecoveryTest, OverriddenZkACLAndCredentialsProvidersTest, OverseerCollectionConfigSetProcessorTest, OverseerStatusTest, PeerSyncReplicationTest]
[junit4] Completed [513/694 (1!)] on J1 in 212.53s, 1 test, 1 failure <<< FAILURES!
[...truncated 63073 lines...]
[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build #
1127 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1127/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseSerialGC
2 tests failed.
FAILED: org.apache.solr.cloud.PeerSyncReplicationTest.test
Error Message:
timeout waiting to see all nodes active
Stack Trace:
java.lang.AssertionError: timeout waiting to see all nodes active
at __randomizedtesting.SeedInfo.seed([2B17085CE3D73B81:A34337864D2B5679]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.PeerSyncReplicationTest.waitTillNodesActive(PeerSyncReplicationTest.java:326)
at org.apache.solr.cloud.PeerSyncReplicationTest.bringUpDeadNodeAndEnsureNoReplication(PeerSyncReplicationTest.java:277)
at org.apache.solr.cloud.PeerSyncReplicationTest.forceNodeFailureAndDoPeerSync(PeerSyncReplicationTest.java:259)
at org.apache.solr.cloud.PeerSyncReplicationTest.test(PeerSyncReplicationTest.java:138)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:745)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation
Error Message:
2 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation: 1) Thread[id=25606, name=jetty-launcher-4016-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037) at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328) at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277) at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105) at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41) at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244) at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44) at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61) at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498) 2) Thread[id=25594, name=jetty-launcher-4016-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037) at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328) at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277) at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105) at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41) at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244) at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44) at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61) at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:
1) Thread[id=25606, name=jetty-launcher-4016-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
2) Thread[id=25594, name=jetty-launcher-4016-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
at __randomizedtesting.SeedInfo.seed([2B17085CE3D73B81]:0)
Build Log:
[...truncated 11629 lines...]
[junit4] Suite: org.apache.solr.cloud.PeerSyncReplicationTest
[junit4] 2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_2B17085CE3D73B81-001/init-core-data-001
[junit4] 2> 976992 INFO (SUITE-PeerSyncReplicationTest-seed#[2B17085CE3D73B81]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields
[junit4] 2> 976994 INFO (SUITE-PeerSyncReplicationTest-seed#[2B17085CE3D73B81]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 976994 INFO (SUITE-PeerSyncReplicationTest-seed#[2B17085CE3D73B81]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 2> 976996 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 976996 INFO (Thread-5217) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 976996 INFO (Thread-5217) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 977096 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.c.ZkTestServer start zk server on port:43857
[junit4] 2> 977113 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 977116 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 977119 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 977121 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 977124 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 977127 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 977129 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 977131 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 977133 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 977136 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 977138 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.c.AbstractZkTestCase put /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 977320 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_2B17085CE3D73B81-001/control-001/cores/collection1
[junit4] 2> 977321 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 977323 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@23e3fd3d{/,null,AVAILABLE}
[junit4] 2> 977323 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@cedde21{HTTP/1.1,[http/1.1]}{127.0.0.1:63992}
[junit4] 2> 977323 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.e.j.s.Server Started @980426ms
[junit4] 2> 977323 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_2B17085CE3D73B81-001/tempDir-001/control/data, hostContext=/, hostPort=63992, coreRootDirectory=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_2B17085CE3D73B81-001/control-001/cores}
[junit4] 2> 977324 ERROR (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 977346 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.0.0
[junit4] 2> 977347 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 977347 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 977347 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-12T15:57:26.098Z
[junit4] 2> 977353 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 977353 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_2B17085CE3D73B81-001/control-001/solr.xml
[junit4] 2> 977361 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:43857/solr
[junit4] 2> 977400 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [n:127.0.0.1:63992_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:63992_
[junit4] 2> 977402 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [n:127.0.0.1:63992_ ] o.a.s.c.Overseer Overseer (id=97446464438206468-127.0.0.1:63992_-n_0000000000) starting
[junit4] 2> 977413 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [n:127.0.0.1:63992_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63992_
[junit4] 2> 977433 INFO (zkCallback-1106-thread-1-processing-n:127.0.0.1:63992_) [n:127.0.0.1:63992_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 977524 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [n:127.0.0.1:63992_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_2B17085CE3D73B81-001/control-001/cores
[junit4] 2> 977524 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [n:127.0.0.1:63992_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 977530 INFO (OverseerStateUpdate-97446464438206468-127.0.0.1:63992_-n_0000000000) [n:127.0.0.1:63992_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 978546 WARN (coreLoadExecutor-3212-thread-1-processing-n:127.0.0.1:63992_) [n:127.0.0.1:63992_ c:control_collection x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 978547 INFO (coreLoadExecutor-3212-thread-1-processing-n:127.0.0.1:63992_) [n:127.0.0.1:63992_ c:control_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 978577 INFO (coreLoadExecutor-3212-thread-1-processing-n:127.0.0.1:63992_) [n:127.0.0.1:63992_ c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 978655 WARN (coreLoadExecutor-3212-thread-1-processing-n:127.0.0.1:63992_) [n:127.0.0.1:63992_ c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 978657 INFO (coreLoadExecutor-3212-thread-1-processing-n:127.0.0.1:63992_) [n:127.0.0.1:63992_ c:control_collection x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 978674 INFO (coreLoadExecutor-3212-thread-1-processing-n:127.0.0.1:63992_) [n:127.0.0.1:63992_ c:control_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 978675 INFO (coreLoadExecutor-3212-thread-1-processing-n:127.0.0.1:63992_) [n:127.0.0.1:63992_ c:control_collection x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_2B17085CE3D73B81-001/control-001/cores/collection1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_2B17085CE3D73B81-001/control-001/cores/collection1/data/]
[junit4] 2> 978675 INFO (coreLoadExecutor-3212-thread-1-processing-n:127.0.0.1:63992_) [n:127.0.0.1:63992_ c:control_collection x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@10729b87
[junit4] 2> 978677 INFO (coreLoadExecutor-3212-thread-1-processing-n:127.0.0.1:63992_) [n:127.0.0.1:63992_ c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=20, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.22016856433611462]
[junit4] 2> 978685 WARN (coreLoadExecutor-3212-thread-1-processing-n:127.0.0.1:63992_) [n:127.0.0.1:63992_ c:control_collection x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 978728 INFO (coreLoadExecutor-3212-thread-1-processing-n:127.0.0.1:63992_) [n:127.0.0.1:63992_ c:control_collection x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 978729 INFO (coreLoadExecutor-3212-thread-1-processing-n:127.0.0.1:63992_) [n:127.0.0.1:63992_ c:control_collection x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=1000 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 978729 INFO (coreLoadExecutor-3212-thread-1-processing-n:127.0.0.1:63992_) [n:127.0.0.1:63992_ c:control_collection x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 978729 INFO (coreLoadExecutor-3212-thread-1-processing-n:127.0.0.1:63992_) [n:127.0.0.1:63992_ c:control_collection x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 978730 INFO (coreLoadExecutor-3212-thread-1-processing-n:127.0.0.1:63992_) [n:127.0.0.1:63992_ c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=42, maxMergeAtOnceExplicit=33, maxMergedSegmentMB=16.3427734375, floorSegmentMB=0.7744140625, forceMergeDeletesPctAllowed=22.55739641565683, segmentsPerTier=42.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 978731 INFO (coreLoadExecutor-3212-thread-1-processing-n:127.0.0.1:63992_) [n:127.0.0.1:63992_ c:control_collection x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5f86462a[collection1] main]
[junit4] 2> 978733 INFO (coreLoadExecutor-3212-thread-1-processing-n:127.0.0.1:63992_) [n:127.0.0.1:63992_ c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 978733 INFO (coreLoadExecutor-3212-thread-1-processing-n:127.0.0.1:63992_) [n:127.0.0.1:63992_ c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 978733 INFO (coreLoadExecutor-3212-thread-1-processing-n:127.0.0.1:63992_) [n:127.0.0.1:63992_ c:control_collection x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 978735 INFO (searcherExecutor-3213-thread-1-processing-n:127.0.0.1:63992_ x:collection1 c:control_collection) [n:127.0.0.1:63992_ c:control_collection x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5f86462a[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 978735 INFO (coreLoadExecutor-3212-thread-1-processing-n:127.0.0.1:63992_) [n:127.0.0.1:63992_ c:control_collection x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559143432832679936
[junit4] 2> 978746 INFO (coreZkRegister-3205-thread-1-processing-n:127.0.0.1:63992_ x:collection1 c:control_collection) [n:127.0.0.1:63992_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 978746 INFO (coreZkRegister-3205-thread-1-processing-n:127.0.0.1:63992_ x:collection1 c:control_collection) [n:127.0.0.1:63992_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 978746 INFO (coreZkRegister-3205-thread-1-processing-n:127.0.0.1:63992_ x:collection1 c:control_collection) [n:127.0.0.1:63992_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:63992/collection1/
[junit4] 2> 978746 INFO (coreZkRegister-3205-thread-1-processing-n:127.0.0.1:63992_ x:collection1 c:control_collection) [n:127.0.0.1:63992_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 978746 INFO (coreZkRegister-3205-thread-1-processing-n:127.0.0.1:63992_ x:collection1 c:control_collection) [n:127.0.0.1:63992_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:63992/collection1/ has no replicas
[junit4] 2> 978752 INFO (coreZkRegister-3205-thread-1-processing-n:127.0.0.1:63992_ x:collection1 c:control_collection) [n:127.0.0.1:63992_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:63992/collection1/ shard1
[junit4] 2> 978904 INFO (coreZkRegister-3205-thread-1-processing-n:127.0.0.1:63992_ x:collection1 c:control_collection) [n:127.0.0.1:63992_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 979044 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 979046 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:43857/solr ready
[junit4] 2> 979046 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 979046 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
[junit4] 2> 979220 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_2B17085CE3D73B81-001/shard-1-001/cores/collection1
[junit4] 2> 979221 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_2B17085CE3D73B81-001/shard-1-001
[junit4] 2> 979221 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 979224 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@35a14a5e{/,null,AVAILABLE}
[junit4] 2> 979224 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@462e6b87{HTTP/1.1,[http/1.1]}{127.0.0.1:56733}
[junit4] 2> 979224 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.e.j.s.Server Started @982328ms
[junit4] 2> 979224 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_2B17085CE3D73B81-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=56733, coreRootDirectory=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_2B17085CE3D73B81-001/shard-1-001/cores}
[junit4] 2> 979224 ERROR (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 979226 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.0.0
[junit4] 2> 979226 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 979226 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 979226 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-12T15:57:27.977Z
[junit4] 2> 979231 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 979231 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_2B17085CE3D73B81-001/shard-1-001/solr.xml
[junit4] 2> 979238 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:43857/solr
[junit4] 2> 979256 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [n:127.0.0.1:56733_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 979266 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [n:127.0.0.1:56733_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56733_
[junit4] 2> 979270 INFO (zkCallback-1110-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 979271 INFO (zkCallback-1106-thread-1-processing-n:127.0.0.1:63992_) [n:127.0.0.1:63992_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 979274 INFO (zkCallback-1116-thread-2-processing-n:127.0.0.1:56733_) [n:127.0.0.1:56733_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 979391 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [n:127.0.0.1:56733_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_2B17085CE3D73B81-001/shard-1-001/cores
[junit4] 2> 979392 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [n:127.0.0.1:56733_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 979396 INFO (OverseerStateUpdate-97446464438206468-127.0.0.1:63992_-n_0000000000) [n:127.0.0.1:63992_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 979508 INFO (zkCallback-1116-thread-2-processing-n:127.0.0.1:56733_) [n:127.0.0.1:56733_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 980410 WARN (coreLoadExecutor-3223-thread-1-processing-n:127.0.0.1:56733_) [n:127.0.0.1:56733_ c:collection1 x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 980410 INFO (coreLoadExecutor-3223-thread-1-processing-n:127.0.0.1:56733_) [n:127.0.0.1:56733_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 980421 INFO (coreLoadExecutor-3223-thread-1-processing-n:127.0.0.1:56733_) [n:127.0.0.1:56733_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 980510 WARN (coreLoadExecutor-3223-thread-1-processing-n:127.0.0.1:56733_) [n:127.0.0.1:56733_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 980511 INFO (coreLoadExecutor-3223-thread-1-processing-n:127.0.0.1:56733_) [n:127.0.0.1:56733_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 980528 INFO (coreLoadExecutor-3223-thread-1-processing-n:127.0.0.1:56733_) [n:127.0.0.1:56733_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 980528 INFO (coreLoadExecutor-3223-thread-1-processing-n:127.0.0.1:56733_) [n:127.0.0.1:56733_ c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_2B17085CE3D73B81-001/shard-1-001/cores/collection1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_2B17085CE3D73B81-001/shard-1-001/cores/collection1/data/]
[junit4] 2> 980528 INFO (coreLoadExecutor-3223-thread-1-processing-n:127.0.0.1:56733_) [n:127.0.0.1:56733_ c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@10729b87
[junit4] 2> 980530 INFO (coreLoadExecutor-3223-thread-1-processing-n:127.0.0.1:56733_) [n:127.0.0.1:56733_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=20, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.22016856433611462]
[junit4] 2> 980539 WARN (coreLoadExecutor-3223-thread-1-processing-n:127.0.0.1:56733_) [n:127.0.0.1:56733_ c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 980584 INFO (coreLoadExecutor-3223-thread-1-processing-n:127.0.0.1:56733_) [n:127.0.0.1:56733_ c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 980584 INFO (coreLoadExecutor-3223-thread-1-processing-n:127.0.0.1:56733_) [n:127.0.0.1:56733_ c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=1000 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 980585 INFO (coreLoadExecutor-3223-thread-1-processing-n:127.0.0.1:56733_) [n:127.0.0.1:56733_ c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 980585 INFO (coreLoadExecutor-3223-thread-1-processing-n:127.0.0.1:56733_) [n:127.0.0.1:56733_ c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 980586 INFO (coreLoadExecutor-3223-thread-1-processing-n:127.0.0.1:56733_) [n:127.0.0.1:56733_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=42, maxMergeAtOnceExplicit=33, maxMergedSegmentMB=16.3427734375, floorSegmentMB=0.7744140625, forceMergeDeletesPctAllowed=22.55739641565683, segmentsPerTier=42.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 980587 INFO (coreLoadExecutor-3223-thread-1-processing-n:127.0.0.1:56733_) [n:127.0.0.1:56733_ c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@570e3fa5[collection1] main]
[junit4] 2> 980588 INFO (coreLoadExecutor-3223-thread-1-processing-n:127.0.0.1:56733_) [n:127.0.0.1:56733_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 980589 INFO (coreLoadExecutor-3223-thread-1-processing-n:127.0.0.1:56733_) [n:127.0.0.1:56733_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 980589 INFO (coreLoadExecutor-3223-thread-1-processing-n:127.0.0.1:56733_) [n:127.0.0.1:56733_ c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 980590 INFO (searcherExecutor-3224-thread-1-processing-n:127.0.0.1:56733_ x:collection1 c:collection1) [n:127.0.0.1:56733_ c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@570e3fa5[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 980590 INFO (coreLoadExecutor-3223-thread-1-processing-n:127.0.0.1:56733_) [n:127.0.0.1:56733_ c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559143434777788416
[junit4] 2> 980601 INFO (coreZkRegister-3218-thread-1-processing-n:127.0.0.1:56733_ x:collection1 c:collection1) [n:127.0.0.1:56733_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 980601 INFO (coreZkRegister-3218-thread-1-processing-n:127.0.0.1:56733_ x:collection1 c:collection1) [n:127.0.0.1:56733_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 980601 INFO (coreZkRegister-3218-thread-1-processing-n:127.0.0.1:56733_ x:collection1 c:collection1) [n:127.0.0.1:56733_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:56733/collection1/
[junit4] 2> 980601 INFO (coreZkRegister-3218-thread-1-processing-n:127.0.0.1:56733_ x:collection1 c:collection1) [n:127.0.0.1:56733_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 980602 INFO (coreZkRegister-3218-thread-1-processing-n:127.0.0.1:56733_ x:collection1 c:collection1) [n:127.0.0.1:56733_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:56733/collection1/ has no replicas
[junit4] 2> 980609 INFO (coreZkRegister-3218-thread-1-processing-n:127.0.0.1:56733_ x:collection1 c:collection1) [n:127.0.0.1:56733_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:56733/collection1/ shard1
[junit4] 2> 980716 INFO (zkCallback-1116-thread-2-processing-n:127.0.0.1:56733_) [n:127.0.0.1:56733_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 980765 INFO (coreZkRegister-3218-thread-1-processing-n:127.0.0.1:56733_ x:collection1 c:collection1) [n:127.0.0.1:56733_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 980871 INFO (zkCallback-1116-thread-2-processing-n:127.0.0.1:56733_) [n:127.0.0.1:56733_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 980939 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_2B17085CE3D73B81-001/shard-2-001/cores/collection1
[junit4] 2> 980939 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_2B17085CE3D73B81-001/shard-2-001
[junit4] 2> 980945 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 980947 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7d722fe9{/,null,AVAILABLE}
[junit4] 2> 980947 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3e8e4374{HTTP/1.1,[http/1.1]}{127.0.0.1:47182}
[junit4] 2> 980947 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.e.j.s.Server Started @984051ms
[junit4] 2> 980947 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_2B17085CE3D73B81-001/tempDir-001/jetty2, solrconfig=solrconfig.xml, hostContext=/, hostPort=47182, coreRootDirectory=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_2B17085CE3D73B81-001/shard-2-001/cores}
[junit4] 2> 980947 ERROR (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 980948 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.0.0
[junit4] 2> 980949 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 980949 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 980949 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-12T15:57:29.700Z
[junit4] 2> 980957 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 980957 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_2B17085CE3D73B81-001/shard-2-001/solr.xml
[junit4] 2> 980969 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:43857/solr
[junit4] 2> 981000 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [n:127.0.0.1:47182_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 981009 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [n:127.0.0.1:47182_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47182_
[junit4] 2> 981012 INFO (zkCallback-1106-thread-1-processing-n:127.0.0.1:63992_) [n:127.0.0.1:63992_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 981013 INFO (zkCallback-1110-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 981013 INFO (zkCallback-1116-thread-2-processing-n:127.0.0.1:56733_) [n:127.0.0.1:56733_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 981014 INFO (zkCallback-1122-thread-1-processing-n:127.0.0.1:47182_) [n:127.0.0.1:47182_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 981120 INFO (zkCallback-1116-thread-2-processing-n:127.0.0.1:56733_) [n:127.0.0.1:56733_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 981198 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [n:127.0.0.1:47182_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_2B17085CE3D73B81-001/shard-2-001/cores
[junit4] 2> 981198 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [n:127.0.0.1:47182_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 981201 INFO (OverseerStateUpdate-97446464438206468-127.0.0.1:63992_-n_0000000000) [n:127.0.0.1:63992_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 981306 INFO (zkCallback-1116-thread-2-processing-n:127.0.0.1:56733_) [n:127.0.0.1:56733_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 981306 INFO (zkCallback-1122-thread-1-processing-n:127.0.0.1:47182_) [n:127.0.0.1:47182_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 982222 WARN (coreLoadExecutor-3234-thread-1-processing-n:127.0.0.1:47182_) [n:127.0.0.1:47182_ c:collection1 x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 982222 INFO (coreLoadExecutor-3234-thread-1-processing-n:127.0.0.1:47182_) [n:127.0.0.1:47182_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 982233 INFO (coreLoadExecutor-3234-thread-1-processing-n:127.0.0.1:47182_) [n:127.0.0.1:47182_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 982323 WARN (coreLoadExecutor-3234-thread-1-processing-n:127.0.0.1:47182_) [n:127.0.0.1:47182_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 982324 INFO (coreLoadExecutor-3234-thread-1-processing-n:127.0.0.1:47182_) [n:127.0.0.1:47182_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 982341 INFO (coreLoadExecutor-3234-thread-1-processing-n:127.0.0.1:47182_) [n:127.0.0.1:47182_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 982341 INFO (coreLoadExecutor-3234-thread-1-processing-n:127.0.0.1:47182_) [n:127.0.0.1:47182_ c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_2B17085CE3D73B81-001/shard-2-001/cores/collection1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_2B17085CE3D73B81-001/shard-2-001/cores/collection1/data/]
[junit4] 2> 982341 INFO (coreLoadExecutor-3234-thread-1-processing-n:127.0.0.1:47182_) [n:127.0.0.1:47182_ c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@10729b87
[junit4] 2> 982343 INFO (coreLoadExecutor-3234-thread-1-processing-n:127.0.0.1:47182_) [n:127.0.0.1:47182_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=20, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.22016856433611462]
[junit4] 2> 982352 WARN (coreLoadExecutor-3234-thread-1-processing-n:127.0.0.1:47182_) [n:127.0.0.1:47182_ c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 982397 INFO (coreLoadExecutor-3234-thread-1-processing-n:127.0.0.1:47182_) [n:127.0.0.1:47182_ c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 982397 INFO (coreLoadExecutor-3234-thread-1-processing-n:127.0.0.1:47182_) [n:127.0.0.1:47182_ c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=1000 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 982398 INFO (coreLoadExecutor-3234-thread-1-processing-n:127.0.0.1:47182_) [n:127.0.0.1:47182_ c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 982398 INFO (coreLoadExecutor-3234-thread-1-processing-n:127.0.0.1:47182_) [n:127.0.0.1:47182_ c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 982398 INFO (coreLoadExecutor-3234-thread-1-processing-n:127.0.0.1:47182_) [n:127.0.0.1:47182_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=42, maxMergeAtOnceExplicit=33, maxMergedSegmentMB=16.3427734375, floorSegmentMB=0.7744140625, forceMergeDeletesPctAllowed=22.55739641565683, segmentsPerTier=42.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 982399 INFO (coreLoadExecutor-3234-thread-1-processing-n:127.0.0.1:47182_) [n:127.0.0.1:47182_ c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@364e1c54[collection1] main]
[junit4] 2> 982400 INFO (coreLoadExecutor-3234-thread-1-processing-n:127.0.0.1:47182_) [n:127.0.0.1:47182_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 982401 INFO (coreLoadExecutor-3234-thread-1-processing-n:127.0.0.1:47182_) [n:127.0.0.1:47182_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 982401 INFO (coreLoadExecutor-3234-thread-1-processing-n:127.0.0.1:47182_) [n:127.0.0.1:47182_ c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 982402 INFO (searcherExecutor-3235-thread-1-processing-n:127.0.0.1:47182_ x:collection1 c:collection1) [n:127.0.0.1:47182_ c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@364e1c54[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 982403 INFO (coreLoadExecutor-3234-thread-1-processing-n:127.0.0.1:47182_) [n:127.0.0.1:47182_ c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559143436678856704
[junit4] 2> 982408 INFO (coreZkRegister-3229-thread-1-processing-n:127.0.0.1:47182_ x:collection1 c:collection1) [n:127.0.0.1:47182_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ZkController Core needs to recover:collection1
[junit4] 2> 982408 INFO (updateExecutor-1119-thread-1-processing-n:127.0.0.1:47182_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:47182_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.DefaultSolrCoreState Running recovery
[junit4] 2> 982409 INFO (recoveryExecutor-1120-thread-1-processing-n:127.0.0.1:47182_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:47182_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
[junit4] 2> 982409 INFO (recoveryExecutor-1120-thread-1-processing-n:127.0.0.1:47182_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:47182_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
[junit4] 2> 982409 INFO (recoveryExecutor-1120-thread-1-processing-n:127.0.0.1:47182_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:47182_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1]
[junit4] 2> 982409 INFO (recoveryExecutor-1120-thread-1-processing-n:127.0.0.1:47182_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:47182_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 982409 INFO (recoveryExecutor-1120-thread-1-processing-n:127.0.0.1:47182_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:47182_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Publishing state of core [collection1] as recovering, leader is [http://127.0.0.1:56733/collection1/] and I am [http://127.0.0.1:47182/collection1/]
[junit4] 2> 982415 INFO (recoveryExecutor-1120-thread-1-processing-n:127.0.0.1:47182_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:47182_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:56733]; [WaitForState: action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1:47182_&coreNodeName=core_node2&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
[junit4] 2> 982419 INFO (qtp2057173749-10726) [n:127.0.0.1:56733_ ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node2, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
[junit4] 2> 982419 INFO (qtp2057173749-10726) [n:127.0.0.1:56733_ ] o.a.s.h.a.PrepRecoveryOp Will wait a max of 183 seconds to see collection1 (shard1 of collection1) have state: recovering
[junit4] 2> 982419 INFO (qtp2057173749-10726) [n:127.0.0.1:56733_ ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:47182_, coreNodeName=core_node2, onlyIfActiveCheckResult=false, nodeProps: core_node2:{"core":"collection1","base_url":"http://127.0.0.1:47182","node_name":"127.0.0.1:47182_","state":"down"}
[junit4] 2> 982516 INFO (zkCallback-1122-thread-1-processing-n:127.0.0.1:47182_) [n:127.0.0.1:47182_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 982516 INFO (zkCallback-1116-thread-2-processing-n:127.0.0.1:56733_) [n:127.0.0.1:56733_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 982660 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_2B17085CE3D73B81-001/shard-3-001/cores/collection1
[junit4] 2> 982660 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_2B17085CE3D73B81-001/shard-3-001
[junit4] 2> 982661 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 982663 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@50bfaa93{/,null,AVAILABLE}
[junit4] 2> 982663 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@41d03ebd{HTTP/1.1,[http/1.1]}{127.0.0.1:51172}
[junit4] 2> 982663 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.e.j.s.Server Started @985768ms
[junit4] 2> 982663 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_2B17085CE3D73B81-001/tempDir-001/jetty3, solrconfig=solrconfig.xml, hostContext=/, hostPort=51172, coreRootDirectory=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_2B17085CE3D73B81-001/shard-3-001/cores}
[junit4] 2> 982663 ERROR (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 982665 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.0.0
[junit4] 2> 982665 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 982665 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 982665 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-02-12T15:57:31.416Z
[junit4] 2> 982669 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 982669 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_2B17085CE3D73B81-001/shard-3-001/solr.xml
[junit4] 2> 982675 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:43857/solr
[junit4] 2> 982690 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [n:127.0.0.1:51172_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 982697 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [n:127.0.0.1:51172_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51172_
[junit4] 2> 982699 INFO (zkCallback-1116-thread-2-processing-n:127.0.0.1:56733_) [n:127.0.0.1:56733_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 982701 INFO (zkCallback-1110-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 982701 INFO (zkCallback-1122-thread-1-processing-n:127.0.0.1:47182_) [n:127.0.0.1:47182_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 982704 INFO (zkCallback-1129-thread-1-processing-n:127.0.0.1:51172_) [n:127.0.0.1:51172_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 982704 INFO (zkCallback-1106-thread-1-processing-n:127.0.0.1:63992_) [n:127.0.0.1:63992_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 982813 INFO (zkCallback-1122-thread-2-processing-n:127.0.0.1:47182_) [n:127.0.0.1:47182_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 982814 INFO (zkCallback-1116-thread-2-processing-n:127.0.0.1:56733_) [n:127.0.0.1:56733_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 982831 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [n:127.0.0.1:51172_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_2B17085CE3D73B81-001/shard-3-001/cores
[junit4] 2> 982832 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [n:127.0.0.1:51172_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 982836 INFO (OverseerStateUpdate-97446464438206468-127.0.0.1:63992_-n_0000000000) [n:127.0.0.1:63992_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 982939 INFO (zkCallback-1122-thread-2-processing-n:127.0.0.1:47182_) [n:127.0.0.1:47182_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 982939 INFO (zkCallback-1116-thread-2-processing-n:127.0.0.1:56733_) [n:127.0.0.1:56733_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 982939 INFO (zkCallback-1129-thread-2-processing-n:127.0.0.1:51172_) [n:127.0.0.1:51172_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 983420 INFO (qtp2057173749-10726) [n:127.0.0.1:56733_ ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:47182_, coreNodeName=core_node2, onlyIfActiveCheckResult=false, nodeProps: core_node2:{"core":"collection1","base_url":"http://127.0.0.1:47182","node_name":"127.0.0.1:47182_","state":"recovering"}
[junit4] 2> 983420 INFO (qtp2057173749-10726) [n:127.0.0.1:56733_ ] o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node2, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4] 2> 983420 INFO (qtp2057173749-10726) [n:127.0.0.1:56733_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:47182_&onlyIfLeaderActive=true&core=collection1&coreNodeName=core_node2&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=1002
[junit4] 2> 983856 WARN (coreLoadExecutor-3245-thread-1-processing-n:127.0.0.1:51172_) [n:127.0.0.1:51172_ c:collection1 x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 983857 INFO (coreLoadExecutor-3245-thread-1-processing-n:127.0.0.1:51172_) [n:127.0.0.1:51172_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 983874 INFO (coreLoadExecutor-3245-thread-1-processing-n:127.0.0.1:51172_) [n:127.0.0.1:51172_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 983982 WARN (coreLoadExecutor-3245-thread-1-processing-n:127.0.0.1:51172_) [n:127.0.0.1:51172_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] default search field in schema is text. WARNING: Deprecated, please use 'df' on request instead.
[junit4] 2> 983984 INFO (coreLoadExecutor-3245-thread-1-processing-n:127.0.0.1:51172_) [n:127.0.0.1:51172_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 984000 INFO (coreLoadExecutor-3245-thread-1-processing-n:127.0.0.1:51172_) [n:127.0.0.1:51172_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 984000 INFO (coreLoadExecutor-3245-thread-1-processing-n:127.0.0.1:51172_) [n:127.0.0.1:51172_ c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_2B17085CE3D73B81-001/shard-3-001/cores/collection1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/../../../../../../../../../../export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.PeerSyncReplicationTest_2B17085CE3D73B81-001/shard-3-001/cores/collection1/data/]
[junit4] 2> 984000 INFO (coreLoadExecutor-3245-thread-1-processing-n:127.0.0.1:51172_) [n:127.0.0.1:51172_ c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@10729b87
[junit4] 2> 984002 INFO (coreLoadExecutor-3245-thread-1-processing-n:127.0.0.1:51172_) [n:127.0.0.1:51172_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=20, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.22016856433611462]
[junit4] 2> 984010 WARN (coreLoadExecutor-3245-thread-1-processing-n:127.0.0.1:51172_) [n:127.0.0.1:51172_ c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 984089 INFO (coreLoadExecutor-3245-thread-1-processing-n:127.0.0.1:51172_) [n:127.0.0.1:51172_ c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 984090 INFO (coreLoadExecutor-3245-thread-1-processing-n:127.0.0.1:51172_) [n:127.0.0.1:51172_ c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=1000 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 984090 INFO (coreLoadExecutor-3245-thread-1-processing-n:127.0.0.1:51172_) [n:127.0.0.1:51172_ c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 984090 INFO (coreLoadExecutor-3245-thread-1-processing-n:127.0.0.1:51172_) [n:127.0.0.1:51172_ c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 984091 INFO (coreLoadExecutor-3245-thread-1-processing-n:127.0.0.1:51172_) [n:127.0.0.1:51172_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=42, maxMergeAtOnceExplicit=33, maxMergedSegmentMB=16.3427734375, floorSegmentMB=0.7744140625, forceMergeDeletesPctAllowed=22.55739641565683, segmentsPerTier=42.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 984092 INFO (coreLoadExecutor-3245-thread-1-processing-n:127.0.0.1:51172_) [n:127.0.0.1:51172_ c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@758e293b[collection1] main]
[junit4] 2> 984093 INFO (coreLoadExecutor-3245-thread-1-processing-n:127.0.0.1:51172_) [n:127.0.0.1:51172_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 984094 INFO (coreLoadExecutor-3245-thread-1-processing-n:127.0.0.1:51172_) [n:127.0.0.1:51172_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 984094 INFO (coreLoadExecutor-3245-thread-1-processing-n:127.0.0.1:51172_) [n:127.0.0.1:51172_ c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 984095 INFO (searcherExecutor-3246-thread-1-processing-n:127.0.0.1:51172_ x:collection1 c:collection1) [n:127.0.0.1:51172_ c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@758e293b[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 984095 INFO (coreLoadExecutor-3245-thread-1-processing-n:127.0.0.1:51172_) [n:127.0.0.1:51172_ c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1559143438453047296
[junit4] 2> 984101 INFO (coreZkRegister-3240-thread-1-processing-n:127.0.0.1:51172_ x:collection1 c:collection1) [n:127.0.0.1:51172_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.ZkController Core needs to recover:collection1
[junit4] 2> 984101 INFO (updateExecutor-1126-thread-1-processing-n:127.0.0.1:51172_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:51172_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.DefaultSolrCoreState Running recovery
[junit4] 2> 984102 INFO (recoveryExecutor-1127-thread-1-processing-n:127.0.0.1:51172_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:51172_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
[junit4] 2> 984102 INFO (recoveryExecutor-1127-thread-1-processing-n:127.0.0.1:51172_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:51172_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
[junit4] 2> 984102 INFO (recoveryExecutor-1127-thread-1-processing-n:127.0.0.1:51172_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:51172_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1]
[junit4] 2> 984102 INFO (recoveryExecutor-1127-thread-1-processing-n:127.0.0.1:51172_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:51172_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 984102 INFO (recoveryExecutor-1127-thread-1-processing-n:127.0.0.1:51172_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:51172_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Publishing state of core [collection1] as recovering, leader is [http://127.0.0.1:56733/collection1/] and I am [http://127.0.0.1:51172/collection1/]
[junit4] 2> 984105 INFO (recoveryExecutor-1127-thread-1-processing-n:127.0.0.1:51172_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:51172_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:56733]; [WaitForState: action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1:51172_&coreNodeName=core_node3&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
[junit4] 2> 984108 INFO (qtp2057173749-10732) [n:127.0.0.1:56733_ ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
[junit4] 2> 984108 INFO (qtp2057173749-10732) [n:127.0.0.1:56733_ ] o.a.s.h.a.PrepRecoveryOp Will wait a max of 183 seconds to see collection1 (shard1 of collection1) have state: recovering
[junit4] 2> 984108 INFO (qtp2057173749-10732) [n:127.0.0.1:56733_ ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:51172_, coreNodeName=core_node3, onlyIfActiveCheckResult=false, nodeProps: core_node3:{"core":"collection1","base_url":"http://127.0.0.1:51172","node_name":"127.0.0.1:51172_","state":"down"}
[junit4] 2> 984139 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.SolrTestCaseJ4 ###Starting test
[junit4] 2> 984139 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Wait for recoveries to finish - wait 30 for each attempt
[junit4] 2> 984139 INFO (TEST-PeerSyncReplicationTest.test-seed#[2B17085CE3D73B81]) [ ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):30
[junit4] 2> 984213 INFO (zkCallback-1122-thread-2-processing-n:127.0.0.1:47182_) [n:127.0.0.1:47182_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 984213 INFO (zkCallback-1116-thread-2-processing-n:127.0.0.1:56733_) [n:127.0.0.1:56733_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 984213 INFO (zkCallback-1129-thread-2-processing-n:127.0.0.1:51172_) [n:127.0.0.1:51172_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 985109 INFO (qtp2057173749-10732) [n:127.0.0.1:56733_ ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:51172_, coreNodeName=core_node3, onlyIfActiveCheckResult=false, nodeProps: core_node3:{"core":"collection1","base_url":"http://127.0.0.1:51172","node_name":"127.0.0.1:51172_","state":"recovering"}
[junit4] 2> 985109 INFO (qtp2057173749-10732) [n:127.0.0.1:56733_ ] o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4] 2> 985109 INFO (qtp2057173749-10732) [n:127.0.0.1:56733_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:51172_&onlyIfLeaderActive=true&core=collection1&coreNodeName=core_node3&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=1002
[junit4] 2> 990421 INFO (recoveryExecutor-1120-thread-1-processing-n:127.0.0.1:47182_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:47182_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Attempting to PeerSync from [http://127.0.0.1:56733/collection1/] - recoveringAfterStartup=[true]
[junit4] 2> 990421 INFO (recoveryExecutor-1120-thread-1-processing-n:127.0.0.1:47182_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:47182_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.PeerSync PeerSync: core=collection1 url=http://127.0.0.1:47182 START replicas=[http://127.0.0.1:56733/collection1/] nUpdates=1000
[junit4] 2> 990424 INFO (qtp2057173749-10733) [n:127.0.0.1:56733_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.IndexFingerprint IndexFingerprint millis:0.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
[junit4] 2> 990424 INFO (qtp2057173749-10733) [n:127.0.0.1:56733_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/get params={distrib=false&qt=/get&getFingerprint=9223372036854775807&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 990425 INFO (recoveryExecutor-1120-thread-1-processing-n:127.0.0.1:47182_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:47182_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.IndexFingerprint IndexFingerprint millis:0.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
[junit4] 2> 990425 INFO (recoveryExecutor-1120-thread-1-processing-n:127.0.0.1:47182_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:47182_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.PeerSync We are already in sync. No need to do a PeerSync
[junit4] 2> 990425 INFO (recoveryExecutor-1120-thread-1-processing-n:127.0.0.1:47182_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:47182_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 990425 INFO (recoveryExecutor-1120-thread-1-processing-n:127.0.0.1:47182_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:47182_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 990426 INFO (recoveryExecutor-1120-thread-1-processing-n:127.0.0.1:47182_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:47182_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 990426 INFO (recoveryExecutor-1120-thread-1-processing-n:127.0.0.1:47182_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:47182_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy PeerSync stage of recovery was successful.
[junit4] 2> 990426 INFO (recoveryExecutor-1120-thread-1-processing-n:127.0.0.1:47182_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:47182_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Replaying updates buffered during PeerSync.
[junit4] 2> 990426 INFO (recoveryExecutor-1120-thread-1-processing-n:127.0.0.1:47182_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:47182_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy No replay needed.
[junit4] 2> 990426 INFO (recoveryExecutor-1120-thread-1-processing-n:127.0.0.1:47182_ x:collection1 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:47182_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.RecoveryStrategy Registering as Active after recovery.
[junit4] 2> 990430 INFO (zkCallback-1116-thread-2-processing-n:127.0.0.1:56733_) [n:127.0.0.1:56733_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 990431 INFO (zkCallback-1122-thread-2-processing-n:127.0.0.1:47182_) [n:127.0.0.1:47182_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 990431 INFO (zkCallback-1129-thread-2-processing-n:127.0.0.1:51172_) [n:127.0.0.1:51172_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 992110 INFO (recoveryExecutor-1127-thread-1-processing-n:127.0.0.1:51172_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:51172_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Attempting to PeerSync from [http://127.0.0.1:56733/collection1/] - recoveringAfterStartup=[true]
[junit4] 2> 992110 INFO (recoveryExecutor-1127-thread-1-processing-n:127.0.0.1:51172_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:51172_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.PeerSync PeerSync: core=collection1 url=http://127.0.0.1:51172 START replicas=[http://127.0.0.1:56733/collection1/] nUpdates=1000
[junit4] 2> 992112 INFO (qtp2057173749-10728) [n:127.0.0.1:56733_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.u.IndexFingerprint IndexFingerprint millis:0.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
[junit4] 2> 992112 INFO (qtp2057173749-10728) [n:127.0.0.1:56733_ c:collection1 s:shard1 r:core_node1 x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/get params={distrib=false&qt=/get&getFingerprint=9223372036854775807&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 992113 INFO (recoveryExecutor-1127-thread-1-processing-n:127.0.0.1:51172_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:51172_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.IndexFingerprint IndexFingerprint millis:0.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
[junit4] 2> 992113 INFO (recoveryExecutor-1127-thread-1-processing-n:127.0.0.1:51172_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:51172_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.PeerSync We are already in sync. No need to do a PeerSync
[junit4] 2> 992113 INFO (recoveryExecutor-1127-thread-1-processing-n:127.0.0.1:51172_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:51172_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 992113 INFO (recoveryExecutor-1127-thread-1-processing-n:127.0.0.1:51172_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:51172_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 992113 INFO (recoveryExecutor-1127-thread-1-processing-n:127.0.0.1:51172_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:51172_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 992113 INFO (recoveryExecutor-1127-thread-1-processing-n:127.0.0.1:51172_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:51172_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy PeerSync stage of recovery was successful.
[junit4] 2> 992113 INFO (recoveryExecutor-1127-thread-1-processing-n:127.0.0.1:51172_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:51172_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Replaying updates buffered during PeerSync.
[junit4] 2> 992113 INFO (recoveryExecutor-1127-thread-1-processing-n:127.0.0.1:51172_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:51172_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy No replay needed.
[junit4] 2> 992113 INFO (recoveryExecutor-1127-thread-1-processing-n:127.0.0.1:51172_ x:collection1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:51172_ c:collection1 s:shard1 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Registering as Active after recovery.
[junit4] 2> 99
[...truncated too long message...]
egrationTest, ExitableDirectoryReaderTest, TestInfoStreamLogging, ZkNodePropsTest, ClassificationUpdateProcessorIntegrationTest, TestLockTree, HdfsChaosMonkeyNothingIsSafeTest, DirectSolrSpellCheckerTest, TestComplexPhraseLeadingWildcard, TestJmxIntegration, DistributedFacetPivotSmallAdvancedTest, DistributedExpandComponentTest, TestCharFilters, SpatialFilterTest, PingRequestHandlerTest, TestDelegationWithHadoopAuth, TestConfigOverlay, SynonymTokenizerTest, OverseerRolesTest, SortSpecParsingTest, TestCryptoKeys, BinaryUpdateRequestHandlerTest, SolrTestCaseJ4Test, TestDefaultStatsCache, CopyFieldTest, SimplePostToolTest, SolrIndexConfigTest, TestDynamicLoading, TestFastLRUCache, TestExclusionRuleCollectionAccess, TestSchemaResource, MultiTermTest, TestSolrQueryParserDefaultOperatorResource, IndexSchemaRuntimeFieldTest, TestHighFrequencyDictionaryFactory, QueryEqualityTest, TestChildDocTransformer, ReplicaListTransformerTest, AnalysisAfterCoreReloadTest, MoreLikeThisHandlerTest, AliasIntegrationTest, CurrencyFieldXmlFileTest, AddBlockUpdateTest, TestCSVLoader, TestFaceting, TestSizeLimitedDistributedMap, TestPerFieldSimilarityWithDefaultOverride, TestConfigSetProperties, AsyncCallRequestStatusResponseTest, TestCoreContainer, QueryElevationComponentTest, TestManagedSynonymFilterFactory, JSONWriterTest, TestFunctionQuery, MissingSegmentRecoveryTest, ReplaceNodeTest, CdcrBootstrapTest, HdfsCollectionsAPIDistributedZkTest, HighlighterTest, SolrCLIZkUtilsTest, TestSolrCLIRunExample, SystemInfoHandlerTest, SolrRequestParserTest, SchemaVersionSpecificBehaviorTest, ResponseHeaderTest, DateFieldTest, HLLSerializationTest, TestReRankQParserPlugin, TestBinaryField, DocumentAnalysisRequestHandlerTest, TestStressCloudBlindAtomicUpdates, SimpleMLTQParserTest, TestDefaultSearchFieldResource, StressHdfsTest, TestSolrFieldCacheMBean, UnloadDistributedZkTest, HdfsUnloadDistributedZkTest, TestStressLucene, PeerSyncTest, MergeStrategyTest, TestLeaderElectionWithEmptyReplica, SolrCoreTest, TestDocBasedVersionConstraints, TestSweetSpotSimilarityFactory, BlockJoinFacetSimpleTest, DeleteNodeTest, DocValuesMultiTest, TestNRTOpen, TestShardHandlerFactory, TestExactSharedStatsCache, TestRandomCollapseQParserPlugin, TestStressUserVersions, XmlUpdateRequestHandlerTest, TestLegacyNumericRangeQueryBuilder, BasicDistributedZkTest, OpenCloseCoreStressTest, OverseerTest, ShardRoutingTest, TestReplicationHandler, ClusterStateUpdateTest, LeaderElectionTest, ZkSolrClientTest, TestRandomDVFaceting, ShardRoutingCustomTest, TestDistributedSearch, TestRecovery, TestHashPartitioner, DistributedSpellCheckComponentTest, ZkControllerTest, TestStressReorder, SimpleFacetsTest, SolrCmdDistributorTest, TestLazyCores, SoftAutoCommitTest, TestPseudoReturnFields, TestWordDelimiterFilterFactory, DistanceFunctionTest, TestSolrDeletionPolicy1, DebugComponentTest, DisMaxRequestHandlerTest, PrimitiveFieldTypeTest, TermsComponentTest, TestValueSourceCache, TestIndexingPerformance, TestSolrQueryParser, SolrPluginUtilsTest, TestCollationField, TestCSVResponseWriter, JsonLoaderTest, CSVRequestHandlerTest, TestLFUCache, SpellPossibilityIteratorTest, TestDFRSimilarityFactory, TestPerFieldSimilarity, TestLMJelinekMercerSimilarityFactory, TimeZoneUtilsTest, ScriptEngineTest, URLClassifyProcessorTest, PreAnalyzedFieldTest, PrimUtilsTest, DOMUtilTest, TestSolrJ, TestUtils, TestDocumentBuilder, SliceStateTest, FileUtilsTest, CircularListTest, DistributedMLTComponentTest, TestRTGBase, CursorPagingTest, CollectionStateFormat2Test, CollectionTooManyReplicasTest, ConfigSetsAPITest, ConnectionManagerTest, CreateCollectionCleanupTest, CustomCollectionTest, DistribCursorPagingTest, HttpPartitionTest, LeaderFailureAfterFreshStartTest, RecoveryAfterSoftCommitTest, SaslZkACLProviderTest, ShardSplitTest, TestReplicaProperties, TestSegmentSorting, TestSolrCloudWithSecureImpersonation]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestSolrCloudWithSecureImpersonation -Dtests.seed=2B17085CE3D73B81 -Dtests.slow=true -Dtests.locale=ar-QA -Dtests.timezone=Africa/Asmera -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J0 | TestSolrCloudWithSecureImpersonation (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:
[junit4] > 1) Thread[id=25606, name=jetty-launcher-4016-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
[junit4] > at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
[junit4] > at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
[junit4] > at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
[junit4] > at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] > 2) Thread[id=25594, name=jetty-launcher-4016-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
[junit4] > at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
[junit4] > at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
[junit4] > at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
[junit4] > at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
[junit4] > at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
[junit4] > at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] > at __randomizedtesting.SeedInfo.seed([2B17085CE3D73B81]:0)
[junit4] Completed [530/694 (2!)] on J0 in 38.33s, 12 tests, 1 error <<< FAILURES!
[...truncated 62750 lines...]
-documentation-lint:
[jtidy] Checking for broken html (such as invalid tags)...
[delete] Deleting directory /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/build/jtidy_tmp
[echo] Checking for broken links...
[exec]
[exec] Crawl/parse...
[exec]
[exec] Verify...
[echo] Checking for malformed docs...
[exec]
[exec] /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/docs/solr-solrj/overview-summary.html
[exec] missing: org.apache.solr.client.solrj.io.eval
[exec]
[exec] Missing javadocs were found!
BUILD FAILED
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/build.xml:775: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/build.xml:101: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build.xml:654: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build.xml:670: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/common-build.xml:2493: exec returned: 1
Total time: 97 minutes 50 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any