You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2018/03/01 19:58:39 UTC

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 1706 - Unstable!

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1706/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseParallelGC

2 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation

Error Message:
4 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:     1) Thread[id=14020, name=jetty-launcher-2795-thread-1-EventThread, state=WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)    2) Thread[id=14018, name=jetty-launcher-2795-thread-2-EventThread, state=WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)    3) Thread[id=14019, name=jetty-launcher-2795-thread-1-SendThread(127.0.0.1:44826), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)    4) Thread[id=14017, name=jetty-launcher-2795-thread-2-SendThread(127.0.0.1:44826), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 4 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation: 
   1) Thread[id=14020, name=jetty-launcher-2795-thread-1-EventThread, state=WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
   2) Thread[id=14018, name=jetty-launcher-2795-thread-2-EventThread, state=WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
   3) Thread[id=14019, name=jetty-launcher-2795-thread-1-SendThread(127.0.0.1:44826), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
        at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
   4) Thread[id=14017, name=jetty-launcher-2795-thread-2-SendThread(127.0.0.1:44826), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
        at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
	at __randomizedtesting.SeedInfo.seed([CA3465D8D0D03170]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=14019, name=jetty-launcher-2795-thread-1-SendThread(127.0.0.1:44826), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)    2) Thread[id=14017, name=jetty-launcher-2795-thread-2-SendThread(127.0.0.1:44826), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]         at java.lang.Thread.sleep(Native Method)         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=14019, name=jetty-launcher-2795-thread-1-SendThread(127.0.0.1:44826), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
        at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
   2) Thread[id=14017, name=jetty-launcher-2795-thread-2-SendThread(127.0.0.1:44826), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
        at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
	at __randomizedtesting.SeedInfo.seed([CA3465D8D0D03170]:0)




Build Log:
[...truncated 12991 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation
   [junit4]   2> 1331421 INFO  (SUITE-TestSolrCloudWithSecureImpersonation-seed#[CA3465D8D0D03170]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_CA3465D8D0D03170-001/init-core-data-001
   [junit4]   2> 1331422 WARN  (SUITE-TestSolrCloudWithSecureImpersonation-seed#[CA3465D8D0D03170]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
   [junit4]   2> 1331422 INFO  (SUITE-TestSolrCloudWithSecureImpersonation-seed#[CA3465D8D0D03170]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 1331424 INFO  (SUITE-TestSolrCloudWithSecureImpersonation-seed#[CA3465D8D0D03170]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 1331575 INFO  (SUITE-TestSolrCloudWithSecureImpersonation-seed#[CA3465D8D0D03170]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_CA3465D8D0D03170-001/tempDir-001
   [junit4]   2> 1331575 INFO  (SUITE-TestSolrCloudWithSecureImpersonation-seed#[CA3465D8D0D03170]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1331575 INFO  (Thread-3856) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1331575 INFO  (Thread-3856) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1331577 ERROR (Thread-3856) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1331675 INFO  (SUITE-TestSolrCloudWithSecureImpersonation-seed#[CA3465D8D0D03170]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:44826
   [junit4]   2> 1331682 INFO  (zkConnectionManagerCallback-2798-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1331687 INFO  (jetty-launcher-2795-thread-1) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T21:27:37Z, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 1331688 INFO  (jetty-launcher-2795-thread-2) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T21:27:37Z, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 1331688 INFO  (jetty-launcher-2795-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1331688 INFO  (jetty-launcher-2795-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1331689 INFO  (jetty-launcher-2795-thread-1) [    ] o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 1331689 INFO  (jetty-launcher-2795-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@489df574{/solr,null,AVAILABLE}
   [junit4]   2> 1331689 INFO  (jetty-launcher-2795-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@4806856c{HTTP/1.1,[http/1.1]}{127.0.0.1:33182}
   [junit4]   2> 1331689 INFO  (jetty-launcher-2795-thread-1) [    ] o.e.j.s.Server Started @1333657ms
   [junit4]   2> 1331689 INFO  (jetty-launcher-2795-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33182}
   [junit4]   2> 1331690 ERROR (jetty-launcher-2795-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1331690 INFO  (jetty-launcher-2795-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1331690 INFO  (jetty-launcher-2795-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1331690 INFO  (jetty-launcher-2795-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1331690 INFO  (jetty-launcher-2795-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-03-01T19:16:56.429Z
   [junit4]   2> 1331691 INFO  (jetty-launcher-2795-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1331691 INFO  (jetty-launcher-2795-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1331691 INFO  (jetty-launcher-2795-thread-2) [    ] o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 1331692 INFO  (jetty-launcher-2795-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6dce18c3{/solr,null,AVAILABLE}
   [junit4]   2> 1331692 INFO  (jetty-launcher-2795-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@74a58972{HTTP/1.1,[http/1.1]}{127.0.0.1:61851}
   [junit4]   2> 1331692 INFO  (jetty-launcher-2795-thread-2) [    ] o.e.j.s.Server Started @1333660ms
   [junit4]   2> 1331692 INFO  (jetty-launcher-2795-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=61851}
   [junit4]   2> 1331692 ERROR (jetty-launcher-2795-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1331693 INFO  (jetty-launcher-2795-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1331693 INFO  (jetty-launcher-2795-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1331693 INFO  (jetty-launcher-2795-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1331693 INFO  (jetty-launcher-2795-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-03-01T19:16:56.432Z
   [junit4]   2> 1331695 INFO  (zkConnectionManagerCallback-2800-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1331695 INFO  (zkConnectionManagerCallback-2802-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1331696 INFO  (jetty-launcher-2795-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1331696 INFO  (jetty-launcher-2795-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1331722 INFO  (jetty-launcher-2795-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44826/solr
   [junit4]   2> 1331726 INFO  (jetty-launcher-2795-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44826/solr
   [junit4]   2> 1331736 INFO  (zkConnectionManagerCallback-2806-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1331736 INFO  (zkConnectionManagerCallback-2810-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1331741 INFO  (zkConnectionManagerCallback-2814-thread-1-processing-n:127.0.0.1:61851_solr) [n:127.0.0.1:61851_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1331750 INFO  (zkConnectionManagerCallback-2812-thread-1-processing-n:127.0.0.1:33182_solr) [n:127.0.0.1:33182_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1332185 INFO  (jetty-launcher-2795-thread-2) [n:127.0.0.1:61851_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1332185 INFO  (jetty-launcher-2795-thread-1) [n:127.0.0.1:33182_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1332186 INFO  (jetty-launcher-2795-thread-2) [n:127.0.0.1:61851_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:61851_solr
   [junit4]   2> 1332187 INFO  (jetty-launcher-2795-thread-1) [n:127.0.0.1:33182_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33182_solr
   [junit4]   2> 1332187 INFO  (jetty-launcher-2795-thread-2) [n:127.0.0.1:61851_solr    ] o.a.s.c.Overseer Overseer (id=72062113543946245-127.0.0.1:61851_solr-n_0000000000) starting
   [junit4]   2> 1332228 INFO  (zkCallback-2813-thread-1-processing-n:127.0.0.1:61851_solr) [n:127.0.0.1:61851_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1332239 INFO  (zkCallback-2811-thread-1-processing-n:127.0.0.1:33182_solr) [n:127.0.0.1:33182_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1332240 INFO  (jetty-launcher-2795-thread-1) [n:127.0.0.1:33182_solr    ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.security.HttpParamDelegationTokenPlugin
   [junit4]   2> 1332255 INFO  (jetty-launcher-2795-thread-2) [n:127.0.0.1:61851_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:61851_solr
   [junit4]   2> 1332260 INFO  (zkCallback-2811-thread-1-processing-n:127.0.0.1:33182_solr) [n:127.0.0.1:33182_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1332267 INFO  (zkCallback-2813-thread-1-processing-n:127.0.0.1:61851_solr) [n:127.0.0.1:61851_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1332270 WARN  (jetty-launcher-2795-thread-1) [n:127.0.0.1:33182_solr    ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
   [junit4]   2> 1332273 INFO  (jetty-launcher-2795-thread-1) [n:127.0.0.1:33182_solr    ] o.a.s.s.KerberosPlugin Params: {solr.kerberos.impersonator.user.noGroups.hosts=*, solr.kerberos.impersonator.user.bogusGroup.groups=__some_bogus_group, token.valid=30, cookie.domain=127.0.0.1, solr.kerberos.impersonator.user.anyHostAnyUser.groups=*, solr.kerberos.impersonator.user.bogusGroup.hosts=*, solr.kerberos.impersonator.user.anyHostAnyUser.hosts=*, token.validity=36000, solr.kerberos.impersonator.user.noHosts.groups=*, type=kerberos, solr.kerberos.impersonator.user.localHostAnyGroup.groups=*, delegation-token.token-kind=solr-dt, solr.kerberos.impersonator.user.wrongHost.hosts=1.1.1.1.1.1, cookie.path=/, zk-dt-secret-manager.znodeWorkingPath=solr/security/zkdtsm, solr.kerberos.impersonator.user.localHostAnyGroup.hosts=serv1-vm3,localhost,127.0.0.1, signer.secret.provider.zookeeper.path=/token, solr.kerberos.impersonator.user.anyHostUsersGroup.groups=staff, zk-dt-secret-manager.enable=true, solr.kerberos.impersonator.user.wrongHost.groups=*, kerberos.name.rules=DEFAULT, signer.secret.provider=zookeeper, solr.kerberos.impersonator.user.anyHostUsersGroup.hosts=*}
   [junit4]   2> 1332385 INFO  (jetty-launcher-2795-thread-1) [n:127.0.0.1:33182_solr    ] o.a.c.f.i.CuratorFrameworkImpl Starting
   [junit4]   2> 1332412 INFO  (jetty-launcher-2795-thread-1-EventThread) [n:127.0.0.1:33182_solr    ] o.a.c.f.s.ConnectionStateManager State change: CONNECTED
   [junit4]   2> 1332463 WARN  (jetty-launcher-2795-thread-1) [n:127.0.0.1:33182_solr    ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
   [junit4]   2> 1332519 INFO  (jetty-launcher-2795-thread-1) [n:127.0.0.1:33182_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33182.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4df7f04
   [junit4]   2> 1332526 INFO  (jetty-launcher-2795-thread-1) [n:127.0.0.1:33182_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33182.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4df7f04
   [junit4]   2> 1332526 INFO  (jetty-launcher-2795-thread-1) [n:127.0.0.1:33182_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33182.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4df7f04
   [junit4]   2> 1332528 INFO  (jetty-launcher-2795-thread-1) [n:127.0.0.1:33182_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_CA3465D8D0D03170-001/tempDir-001/node1/.
   [junit4]   2> 1332615 INFO  (jetty-launcher-2795-thread-2) [n:127.0.0.1:61851_solr    ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.security.HttpParamDelegationTokenPlugin
   [junit4]   2> 1332615 WARN  (jetty-launcher-2795-thread-2) [n:127.0.0.1:61851_solr    ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
   [junit4]   2> 1332615 INFO  (jetty-launcher-2795-thread-2) [n:127.0.0.1:61851_solr    ] o.a.s.s.KerberosPlugin Params: {solr.kerberos.impersonator.user.noGroups.hosts=*, solr.kerberos.impersonator.user.bogusGroup.groups=__some_bogus_group, token.valid=30, cookie.domain=127.0.0.1, solr.kerberos.impersonator.user.anyHostAnyUser.groups=*, solr.kerberos.impersonator.user.bogusGroup.hosts=*, solr.kerberos.impersonator.user.anyHostAnyUser.hosts=*, token.validity=36000, solr.kerberos.impersonator.user.noHosts.groups=*, type=kerberos, solr.kerberos.impersonator.user.localHostAnyGroup.groups=*, delegation-token.token-kind=solr-dt, solr.kerberos.impersonator.user.wrongHost.hosts=1.1.1.1.1.1, cookie.path=/, zk-dt-secret-manager.znodeWorkingPath=solr/security/zkdtsm, solr.kerberos.impersonator.user.localHostAnyGroup.hosts=serv1-vm3,localhost,127.0.0.1, signer.secret.provider.zookeeper.path=/token, solr.kerberos.impersonator.user.anyHostUsersGroup.groups=staff, zk-dt-secret-manager.enable=true, solr.kerberos.impersonator.user.wrongHost.groups=*, kerberos.name.rules=DEFAULT, signer.secret.provider=zookeeper, solr.kerberos.impersonator.user.anyHostUsersGroup.hosts=*}
   [junit4]   2> 1332616 INFO  (jetty-launcher-2795-thread-2) [n:127.0.0.1:61851_solr    ] o.a.c.f.i.CuratorFrameworkImpl Starting
   [junit4]   2> 1332624 INFO  (jetty-launcher-2795-thread-2-EventThread) [n:127.0.0.1:61851_solr    ] o.a.c.f.s.ConnectionStateManager State change: CONNECTED
   [junit4]   2> 1332651 WARN  (jetty-launcher-2795-thread-2) [n:127.0.0.1:61851_solr    ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
   [junit4]   2> 1332727 INFO  (jetty-launcher-2795-thread-2) [n:127.0.0.1:61851_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_61851.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4df7f04
   [junit4]   2> 1332734 INFO  (jetty-launcher-2795-thread-2) [n:127.0.0.1:61851_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_61851.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4df7f04
   [junit4]   2> 1332734 INFO  (jetty-launcher-2795-thread-2) [n:127.0.0.1:61851_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_61851.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4df7f04
   [junit4]   2> 1332736 INFO  (jetty-launcher-2795-thread-2) [n:127.0.0.1:61851_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_CA3465D8D0D03170-001/tempDir-001/node2/.
   [junit4]   2> 1332772 INFO  (zkConnectionManagerCallback-2820-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1332783 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100041c478d0009, likely client has closed socket
   [junit4]   2> 1332811 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateHost-seed#[CA3465D8D0D03170]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testProxyValidateHost
   [junit4]   2> 1332825 INFO  (qtp614567702-13887) [n:127.0.0.1:33182_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params doAs=bar&action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1332826 INFO  (qtp614567702-13887) [n:127.0.0.1:33182_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={doAs=bar&action=LIST&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1332826 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateHost-seed#[CA3465D8D0D03170]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=localHostAnyGroup&p=localHostAnyGroup&t=test&e=1519967817561&s=vHOJ9bNjQQHt+AP7tY3SrCB4i/8="; Path=/; Domain=127.0.0.1; Expires=   , 02-   -2018 05:16:57 GMT; HttpOnly". Invalid 'expires' attribute: , 02- -2018 05:16:57 GMT
   [junit4]   2> 1332826 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateHost-seed#[CA3465D8D0D03170]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testProxyValidateHost
   [junit4]   2> 1332853 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateGroup-seed#[CA3465D8D0D03170]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testProxyValidateGroup
   [junit4]   2> 1332906 INFO  (qtp614567702-13889) [n:127.0.0.1:33182_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params doAs=jenkins&action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1332906 INFO  (qtp614567702-13889) [n:127.0.0.1:33182_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={doAs=jenkins&action=LIST&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1332906 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateGroup-seed#[CA3465D8D0D03170]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=anyHostUsersGroup&p=anyHostUsersGroup&t=test&e=1519967817593&s=1mOu2WHz6UGIHWS/W60Lgv9Wv84="; Path=/; Domain=127.0.0.1; Expires=   , 02-   -2018 05:16:57 GMT; HttpOnly". Invalid 'expires' attribute: , 02- -2018 05:16:57 GMT
   [junit4]   2> 1332906 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateGroup-seed#[CA3465D8D0D03170]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testProxyValidateGroup
   [junit4]   2> 1332927 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateAnyHostAnyUser-seed#[CA3465D8D0D03170]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testProxyValidateAnyHostAnyUser
   [junit4]   2> 1332929 INFO  (qtp614567702-13891) [n:127.0.0.1:33182_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params doAs=bar&action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1332929 INFO  (qtp614567702-13891) [n:127.0.0.1:33182_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={doAs=bar&action=LIST&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1332929 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateAnyHostAnyUser-seed#[CA3465D8D0D03170]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=anyHostAnyUser&p=anyHostAnyUser&t=test&e=1519967817667&s=MtcUYKo/PdF7xPU3HRu6zVijo7k="; Path=/; Domain=127.0.0.1; Expires=   , 02-   -2018 05:16:57 GMT; HttpOnly". Invalid 'expires' attribute: , 02- -2018 05:16:57 GMT
   [junit4]   2> 1332929 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateAnyHostAnyUser-seed#[CA3465D8D0D03170]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testProxyValidateAnyHostAnyUser
   [junit4]   2> 1332953 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidRemote-seed#[CA3465D8D0D03170]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testProxyInvalidRemote
   [junit4]   2> 1332962 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidRemote-seed#[CA3465D8D0D03170]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=localHostAnyGroup&p=localHostAnyGroup&t=test&e=1519967817692&s=IwiepqXue1t25pNDaLMQBinzq9U="; Path=/; Domain=127.0.0.1; Expires=   , 02-   -2018 05:16:57 GMT; HttpOnly". Invalid 'expires' attribute: , 02- -2018 05:16:57 GMT
   [junit4]   2> 1332963 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidRemote-seed#[CA3465D8D0D03170]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testProxyInvalidRemote
   [junit4]   2> 1332992 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[CA3465D8D0D03170]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testForwarding
   [junit4]   2> 1332994 INFO  (zkConnectionManagerCallback-2823-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1333898 INFO  (zkConnectionManagerCallback-2826-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1333900 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[CA3465D8D0D03170]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 1333901 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[CA3465D8D0D03170]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:44826/solr ready
   [junit4]   2> 1333903 INFO  (qtp654101897-13901) [n:127.0.0.1:61851_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=0&replicationFactor=1&collection.configName=conf1&maxShardsPerNode=1&name=forwardingCollection&nrtReplicas=1&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1333909 INFO  (OverseerThreadFactory-5437-thread-1-processing-n:127.0.0.1:61851_solr) [n:127.0.0.1:61851_solr    ] o.a.s.c.a.c.CreateCollectionCmd Create collection forwardingCollection
   [junit4]   2> 1334036 INFO  (OverseerStateUpdate-72062113543946245-127.0.0.1:61851_solr-n_0000000000) [n:127.0.0.1:61851_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"forwardingCollection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"forwardingCollection_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:61851/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 1334250 INFO  (qtp654101897-13898) [n:127.0.0.1:61851_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf1&newCollection=true&name=forwardingCollection_shard1_replica_n1&action=CREATE&numShards=1&collection=forwardingCollection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1334251 INFO  (qtp654101897-13898) [n:127.0.0.1:61851_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1334376 INFO  (zkCallback-2813-thread-1-processing-n:127.0.0.1:61851_solr) [n:127.0.0.1:61851_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1334377 INFO  (zkCallback-2813-thread-2-processing-n:127.0.0.1:61851_solr) [n:127.0.0.1:61851_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1335296 INFO  (qtp654101897-13898) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1335332 INFO  (qtp654101897-13898) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.IndexSchema [forwardingCollection_shard1_replica_n1] Schema name=test
   [junit4]   2> 1335451 INFO  (qtp654101897-13898) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 1335473 INFO  (qtp654101897-13898) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'forwardingCollection_shard1_replica_n1' using configuration from collection forwardingCollection, trusted=true
   [junit4]   2> 1335474 INFO  (qtp654101897-13898) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_61851.solr.core.forwardingCollection.shard1.replica_n1' (registry 'solr.core.forwardingCollection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4df7f04
   [junit4]   2> 1335474 INFO  (qtp654101897-13898) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1335474 INFO  (qtp654101897-13898) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore [[forwardingCollection_shard1_replica_n1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_CA3465D8D0D03170-001/tempDir-001/node2/forwardingCollection_shard1_replica_n1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_CA3465D8D0D03170-001/tempDir-001/node2/./forwardingCollection_shard1_replica_n1/data/]
   [junit4]   2> 1335476 INFO  (qtp654101897-13898) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=169819254, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 1335668 INFO  (qtp654101897-13898) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1335668 INFO  (qtp654101897-13898) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1335675 INFO  (qtp654101897-13898) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1335675 INFO  (qtp654101897-13898) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1335680 INFO  (qtp654101897-13898) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=29, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5203224028566534]
   [junit4]   2> 1335680 INFO  (qtp654101897-13898) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@70253758[forwardingCollection_shard1_replica_n1] main]
   [junit4]   2> 1335682 INFO  (qtp654101897-13898) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 1335801 INFO  (qtp654101897-13898) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 1336085 INFO  (qtp654101897-13898) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1336085 INFO  (qtp654101897-13898) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1593764028952346624
   [junit4]   2> 1336087 INFO  (searcherExecutor-5442-thread-1-processing-n:127.0.0.1:61851_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 1336089 INFO  (searcherExecutor-5442-thread-1-processing-n:127.0.0.1:61851_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 1336090 INFO  (searcherExecutor-5442-thread-1-processing-n:127.0.0.1:61851_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 1336098 INFO  (searcherExecutor-5442-thread-1-processing-n:127.0.0.1:61851_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 1336103 INFO  (searcherExecutor-5442-thread-1-processing-n:127.0.0.1:61851_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 1336106 INFO  (qtp654101897-13898) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1336106 INFO  (qtp654101897-13898) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1336106 INFO  (qtp654101897-13898) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:61851/solr/forwardingCollection_shard1_replica_n1/
   [junit4]   2> 1336106 INFO  (qtp654101897-13898) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1336106 INFO  (qtp654101897-13898) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:61851/solr/forwardingCollection_shard1_replica_n1/ has no replicas
   [junit4]   2> 1336106 INFO  (qtp654101897-13898) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 1336109 INFO  (searcherExecutor-5442-thread-1-processing-n:127.0.0.1:61851_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 1336112 INFO  (qtp654101897-13898) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:61851/solr/forwardingCollection_shard1_replica_n1/ shard1
   [junit4]   2> 1336114 INFO  (searcherExecutor-5442-thread-1-processing-n:127.0.0.1:61851_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 1336114 INFO  (searcherExecutor-5442-thread-1-processing-n:127.0.0.1:61851_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 1336114 INFO  (searcherExecutor-5442-thread-1-processing-n:127.0.0.1:61851_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 1336115 INFO  (searcherExecutor-5442-thread-1-processing-n:127.0.0.1:61851_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore [forwardingCollection_shard1_replica_n1] Registered new searcher Searcher@70253758[forwardingCollection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1336218 INFO  (zkCallback-2813-thread-3-processing-n:127.0.0.1:61851_solr) [n:127.0.0.1:61851_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1336219 INFO  (zkCallback-2813-thread-2-processing-n:127.0.0.1:61851_solr) [n:127.0.0.1:61851_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1336223 INFO  (qtp654101897-13898) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1336230 INFO  (qtp654101897-13898) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf1&newCollection=true&name=forwardingCollection_shard1_replica_n1&action=CREATE&numShards=1&collection=forwardingCollection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1980
   [junit4]   2> 1336230 WARN  (httpShardExecutor-5433-thread-1-processing-n:127.0.0.1:61851_solr [http:////127.0.0.1:61851//solr] http:////127.0.0.1:61851//solr) [n:127.0.0.1:61851_solr    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=$&p=$&t=test&e=1519967818988&s=pAJ0k0lCHYUypxZVGLzNxQrdEqI="; Path=/; Domain=127.0.0.1; Expires=   , 02-   -2018 05:16:58 GMT; HttpOnly". Invalid 'expires' attribute: , 02- -2018 05:16:58 GMT
   [junit4]   2> 1336235 INFO  (qtp654101897-13901) [n:127.0.0.1:61851_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 1336332 INFO  (zkCallback-2813-thread-2-processing-n:127.0.0.1:61851_solr) [n:127.0.0.1:61851_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1336332 INFO  (zkCallback-2813-thread-3-processing-n:127.0.0.1:61851_solr) [n:127.0.0.1:61851_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1337247 INFO  (qtp654101897-13901) [n:127.0.0.1:61851_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={pullReplicas=0&replicationFactor=1&collection.configName=conf1&maxShardsPerNode=1&name=forwardingCollection&nrtReplicas=1&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2} status=0 QTime=3344
   [junit4]   2> 1337247 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[CA3465D8D0D03170]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=user&p=user&t=test&e=1519967818641&s=wJf472U42BZZikF76oYdVvQw4xY="; Path=/; Domain=127.0.0.1; Expires=   , 02-   -2018 05:16:58 GMT; HttpOnly". Invalid 'expires' attribute: , 02- -2018 05:16:58 GMT
   [junit4]   2> 1337247 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[CA3465D8D0D03170]) [    ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: forwardingCollection failOnTimeout:true timeout (sec):100
   [junit4]   2> 1337253 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[CA3465D8D0D03170]) [    ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: forwardingCollection
   [junit4]   2> 1337256 INFO  (qtp614567702-13889) [n:127.0.0.1:33182_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1337259 INFO  (qtp654101897-13898) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.S.Request [forwardingCollection_shard1_replica_n1]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 1337262 WARN  (qtp614567702-13889) [n:127.0.0.1:33182_solr    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=$&p=$&t=test&e=1519967821997&s=SGkrrLh/pqg0L0gY19YnnquMpIU="; Path=/; Domain=127.0.0.1; Expires=   , 02-   -2018 05:17:01 GMT; HttpOnly". Invalid 'expires' attribute: , 02- -2018 05:17:01 GMT
   [junit4]   2> 1337262 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[CA3465D8D0D03170]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=user&p=user&t=test&e=1519967821995&s=ZbGXPVrmvIaClryULhM6btFpnfs="; Path=/; Domain=127.0.0.1; Expires=   , 02-   -2018 05:17:01 GMT; HttpOnly". Invalid 'expires' attribute: , 02- -2018 05:17:01 GMT
   [junit4]   2> 1337263 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[CA3465D8D0D03170]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=$&p=$&t=test&e=1519967821997&s=SGkrrLh/pqg0L0gY19YnnquMpIU="; Path=/; Domain=127.0.0.1; Expires=   , 02-   -2018 05:17:01 GMT; HttpOnly". Invalid 'expires' attribute: , 02- -2018 05:17:01 GMT
   [junit4]   2> 1337265 INFO  (qtp654101897-13898) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.S.Request [forwardingCollection_shard1_replica_n1]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 1337266 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[CA3465D8D0D03170]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=user&p=user&t=test&e=1519967822004&s=BMdmoRlmdegX5baVCCEMHhOShTw="; Path=/; Domain=127.0.0.1; Expires=   , 02-   -2018 05:17:02 GMT; HttpOnly". Invalid 'expires' attribute: , 02- -2018 05:17:02 GMT
   [junit4]   2> 1337266 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[CA3465D8D0D03170]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testForwarding
   [junit4]   2> 1337294 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidProxyUser-seed#[CA3465D8D0D03170]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testProxyInvalidProxyUser
   [junit4]   2> 1337300 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidProxyUser-seed#[CA3465D8D0D03170]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=bar&p=bar&t=test&e=1519967822035&s=V9j7XsQxcPnDKKGG8iz4tYySQHo="; Path=/; Domain=127.0.0.1; Expires=   , 02-   -2018 05:17:02 GMT; HttpOnly". Invalid 'expires' attribute: , 02- -2018 05:17:02 GMT
   [junit4]   2> 1337301 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidProxyUser-seed#[CA3465D8D0D03170]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testProxyInvalidProxyUser
   [junit4]   2> 1337342 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyNullProxyUser-seed#[CA3465D8D0D03170]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testProxyNullProxyUser
   [junit4]   2> 1337342 INFO  (qtp614567702-13889) [n:127.0.0.1:33182_solr    ] o.a.s.s.SolrDispatchFilter Error authenticating
   [junit4]   2> java.lang.IllegalArgumentException: userName is NULL, empty or contains a '&'
   [junit4]   2> 	at org.apache.hadoop.security.authentication.util.AuthToken.checkForIllegalArgument(AuthToken.java:87)
   [junit4]   2> 	at org.apache.hadoop.security.authentication.util.AuthToken.<init>(AuthToken.java:70)
   [junit4]   2> 	at org.apache.hadoop.security.authentication.server.AuthenticationToken.<init>(AuthenticationToken.java:58)
   [junit4]   2> 	at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamDelegationTokenAuthenticationHandler$HttpParamAuthenticationHandler.authenticate(HttpParamDelegationTokenPlugin.java:213)
   [junit4]   2> 	at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.authenticate(DelegationTokenAuthenticationHandler.java:348)
   [junit4]   2> 	at org.apache.solr.security.RequestContinuesRecorderAuthenticationHandler.authenticate(RequestContinuesRecorderAuthenticationHandler.java:69)
   [junit4]   2> 	at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:518)
   [junit4]   2> 	at org.apache.solr.security.DelegationTokenKerberosFilter.doFilter(DelegationTokenKerberosFilter.java:134)
   [junit4]   2> 	at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamToRequestFilter.doFilter(HttpParamDelegationTokenPlugin.java:245)
   [junit4]   2> 	at org.apache.solr.security.KerberosPlugin.doAuthenticate(KerberosPlugin.java:243)
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:458)
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:353)
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:330)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
   [junit4]   2> 	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
   [junit4]   2> 	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
   [junit4]   2> 	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:527)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
   [junit4]   2> 	at org.eclipse.jetty.server.Server.handle(Server.java:530)
   [junit4]   2> 	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)
   [junit4]   2> 	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256)
   [junit4]   2> 	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
   [junit4]   2> 	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
   [junit4]   2> 	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 1337343 WARN  (qtp614567702-13889) [n:127.0.0.1:33182_solr    ] o.e.j.s.HttpChannel /solr/admin/collections
   [junit4]   2> org.apache.solr.common.SolrException: Error during request authentication, 
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:464)
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:353)
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:330)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
   [junit4]   2> 	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
   [junit4]   2> 	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
   [junit4]   2> 	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:527)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
   [junit4]   2> 	at org.eclipse.jetty.server.Server.handle(Server.java:530)
   [junit4]   2> 	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)
   [junit4]   2> 	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256)
   [junit4]   2> 	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
   [junit4]   2> 	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
   [junit4]   2> 	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
   [junit4]   2> 	at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> Caused by: java.lang.IllegalArgumentException: userName is NULL, empty or contains a '&'
   [junit4]   2> 	at org.apache.hadoop.security.authentication.util.AuthToken.checkForIllegalArgument(AuthToken.java:87)
   [junit4]   2> 	at org.apache.hadoop.security.authentication.util.AuthToken.<init>(AuthToken.java:70)
   [junit4]   2> 	at org.apache.hadoop.security.authentication.server.AuthenticationToken.<init>(AuthenticationToken.java:58)
   [junit4]   2> 	at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamDelegationTokenAuthenticationHandler$HttpParamAuthenticationHandler.authenticate(HttpParamDelegationTokenPlugin.java:213)
   [junit4]   2> 	at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.authenticate(DelegationTokenAuthenticationHandler.java:348)
   [junit4]   2> 	at org.apache.solr.security.RequestContinuesRecorderAuthenticationHandler.authenticate(RequestContinuesRecorderAuthenticationHandler.java:69)
   [junit4]   2> 	at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:518)
   [junit4]   2> 	at org.apache.solr.security.DelegationTokenKerberosFilter.doFilter(DelegationTokenKerberosFilter.java:134)
   [junit4]   2> 	at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamToRequestFilter.doFilter(HttpParamDelegationTokenPlugin.java:245)
   [junit4]   2> 	at org.apache.solr.security.KerberosPlugin.doAuthenticate(KerberosPlugin.java:243)
   [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:458)
   [junit4]   2> 	... 31 more
   [junit4]   2> 1337346 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyNullProxyUser-seed#[CA3465D8D0D03170]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testProxyNullProxyUser
   [junit4]   2> 1337369 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigHosts-seed#[CA3465D8D0D03170]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testProxyNoConfigHosts
   [junit4]   2> 1337371 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigHosts-seed#[CA3465D8D0D03170]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=noHosts&p=noHosts&t=test&e=1519967822109&s=k+KLjVRTVmRprQykfNlDifCddY0="; Path=/; Domain=127.0.0.1; Expires=   , 02-   -2018 05:17:02 GMT; HttpOnly". Invalid 'expires' attribute: , 02- -2018 05:17:02 GMT
   [junit4]   2> 1337372 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigHosts-seed#[CA3465D8D0D03170]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testProxyNoConfigHosts
   [junit4]   2> 1337470 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidGroup-seed#[CA3465D8D0D03170]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testProxyInvalidGroup
   [junit4]   2> 1337499 WARN  (qtp614567702-13889) [n:127.0.0.1:33182_solr    ] o.a.h.s.ShellBasedUnixGroupsMapping got exception trying to get groups for user bar: id: invalid user name: "bar"
   [junit4]   2> id: invalid user name: "bar"
   [junit4]   2> 
   [junit4]   2> 1337501 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidGroup-seed#[CA3465D8D0D03170]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=bogusGroup&p=bogusGroup&t=test&e=1519967822209&s=kuwoweoQfie1NuZwWLlrEItMTDw="; Path=/; Domain=127.0.0.1; Expires=   , 02-   -2018 05:17:02 GMT; HttpOnly". Invalid 'expires' attribute: , 02- -2018 05:17:02 GMT
   [junit4]   2> 1337501 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidGroup-seed#[CA3465D8D0D03170]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testProxyInvalidGroup
   [junit4]   2> 1337519 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyUnknownRemote-seed#[CA3465D8D0D03170]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testProxyUnknownRemote
   [junit4]   2> 1354901 WARN  (jetty-launcher-2795-thread-1-SendThread(127.0.0.1:44826)) [n:127.0.0.1:33182_solr    ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 17647ms for sessionid 0x100041c478d0006
   [junit4]   2> 1354902 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100041c478d0006, likely client has closed socket
   [junit4]   2> 1354902 WARN  (jetty-launcher-2795-thread-2-SendThread(127.0.0.1:44826)) [n:127.0.0.1:61851_solr    ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 18572ms for sessionid 0x100041c478d0005
   [junit4]   2> 1354902 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testProxyUnknownRemote-seed#[CA3465D8D0D03170]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=localHostAnyGroup&p=localHostAnyGroup&t=test&e=1519967822258&s=LCjf1y0j81wcCxhztMsO52Q81dU="; Path=/; Domain=127.0.0.1; Expires=   , 02-   -2018 05:17:02 GMT; HttpOnly". Invalid 'expires' attribute: , 02- -2018 05:17:02 GMT
   [junit4]   2> 1354903 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100041c478d0005, likely client has closed socket
   [junit4]   2> 1354903 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyUnknownRemote-seed#[CA3465D8D0D03170]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testProxyUnknownRemote
   [junit4]   2> 1354924 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigGroups-seed#[CA3465D8D0D03170]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testProxyNoConfigGroups
   [junit4]   2> 1354926 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigGroups-seed#[CA3465D8D0D03170]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=noGroups&p=noGroups&t=test&e=1519967839663&s=MfYjnA0Auo93F5BPErrT6oQfipk="; Path=/; Domain=127.0.0.1; Expires=   , 02-   -2018 05:17:19 GMT; HttpOnly". Invalid 'expires' attribute: , 02- -2018 05:17:19 GMT
   [junit4]   2> 1354926 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigGroups-seed#[CA3465D8D0D03170]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testProxyNoConfigGroups
   [junit4]   2> 1354956 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyWrongHost-seed#[CA3465D8D0D03170]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testProxyWrongHost
   [junit4]   2> 1355002 WARN  (zkConnectionManagerCallback-2812-thread-1-processing-n:127.0.0.1:33182_solr) [n:127.0.0.1:33182_solr    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@b851009 name: ZooKeeperConnection Watcher:127.0.0.1:44826/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 1355002 WARN  (zkConnectionManagerCallback-2812-thread-1-processing-n:127.0.0.1:33182_solr) [n:127.0.0.1:33182_solr    ] o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 1355003 WARN  (zkConnectionManagerCallback-2826-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@486bb2cc name: ZooKeeperConnection Watcher:127.0.0.1:44826/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 1355003 WARN  (zkConnectionManagerCallback-2826-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 1355006 INFO  (jetty-launcher-2795-thread-1-EventThread) [n:127.0.0.1:33182_solr    ] o.a.c.f.s.ConnectionStateManager State change: SUSPENDED
   [junit4]   2> 1355006 WARN  (zkConnectionManagerCallback-2814-thread-1-processing-n:127.0.0.1:61851_solr) [n:127.0.0.1:61851_solr    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@787f1d59 name: ZooKeeperConnection Watcher:127.0.0.1:44826/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
   [junit4]   2> 1355007 WARN  (zkConnectionManagerCallback-2814-thread-1-processing-n:127.0.0.1:61851_solr) [n:127.0.0.1:61851_solr    ] o.a.s.c.c.ConnectionManager zkClient has disconnected
   [junit4]   2> 1355008 INFO  (jetty-launcher-2795-thread-2-EventThread) [n:127.0.0.1:61851_solr    ] o.a.c.f.s.ConnectionStateManager State change: SUSPENDED
   [junit4]   2> 1364079 WARN  (jetty-launcher-2795-thread-1-SendThread(127.0.0.1:44826)) [n:127.0.0.1:33182_solr    ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x100041c478d0006 has expired
   [junit4]   2> 1364080 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testProxyWrongHost-seed#[CA3465D8D0D03170]) [    ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=wrongHost&p=wrongHost&t=test&e=1519967839695&s=YOLNsy4lUCPgFCcgjIJbXHzm8Nk="; Path=/; Domain=127.0.0.1; Expires=   , 02-   -2018 05:17:19 GMT; HttpOnly". Invalid 'expires' attribute: , 02- -2018 05:17:19 GMT
   [junit4]   2> 1364080 WARN  (zkConnectionManagerCallback-2812-thread-1-processing-n:127.0.0.1:33182_solr) [n:127.0.0.1:33182_solr    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@b851009 name: ZooKeeperConnection Watcher:127.0.0.1:44826/solr got event WatchedEvent state:Expired type:None path:null path: null type: None
   [junit4]   2> 1364080 INFO  (TEST-TestSolrCloudWithSecureImpersonation.testProxyWrongHost-seed#[CA3465D8D0D03170]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testProxyWrongHost
   [junit4]   2> 1364080 WARN  (zkConnectionManagerCallback-2812-thread-1-processing-n:127.0.0.1:33182_solr) [n:127.0.0.1:33182_solr    ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper...
   [junit4]   2> 1364080 WARN  (zkConnectionManagerCallback-2812-thread-1-processing-n:127.0.0.1:33182_solr) [n:127.0.0.1:33182_solr    ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
   [junit4]   2> 1364088 WARN  (jetty-launcher-2795-thread-1-SendThread(127.0.0.1:44826)) [n:127.0.0.1:33182_solr    ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x100041c478d0007 has expired
   [junit4]   2> 1364082 WARN  (jetty-launcher-2795-thread-2-SendThread(127.0.0.1:44826)) [n:127.0.0.1:61851_solr    ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x100041c478d0008 has expired
   [junit4]   2> 1364082 WARN  (zkConnectionManagerCallback-2826-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@486bb2cc name: ZooKeeperConnection Watcher:127.0.0.1:44826/solr got event WatchedEvent state:Expired type:None path:null path: null type: None
   [junit4]   2> 1364090 WARN  (zkConnectionManagerCallback-2826-thread-1) [    ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper...
   [junit4]   2> 1364090 WARN  (zkConnectionManagerCallback-2826-thread-1) [    ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
   [junit4]   2> 1364082 WARN  (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[CA3465D8D0D03170]-SendThread(127.0.0.1:44826)) [    ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x100041c478d000b has expired
   [junit4]   2> 1364090 INFO  (zkConnectionManagerCallback-2814-thread-1-processing-n:127.0.0.1:61851_solr) [n:127.0.0.1:61851_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1364092 INFO  (OverseerCollectionConfigSetProcessor-72062113543946245-127.0.0.1:61851_solr-n_0000000000) [n:127.0.0.1:61851_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1364103 INFO  (zkConnectionManagerCallback-2812-thread-1-processing-n:127.0.0.1:33182_solr-EventThread) [n:127.0.0.1:33182_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1364104 INFO  (zkCallback-2813-thread-10-processing-n:127.0.0.1:61851_solr) [n:127.0.0.1:61851_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 1364105 INFO  (zkConnectionManagerCallback-2812-thread-1-processing-n:127.0.0.1:33182_solr) [n:127.0.0.1:33182_solr    ] o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
   [junit4]   2> 1364105 INFO  (zkConnectionManagerCallback-2812-thread-1-processing-n:127.0.0.1:33182_solr) [n:127.0.0.1:33182_solr    ] o.a.s.c.ZkController ZooKeeper session re-connected ... refreshing core states after session expiration.
   [junit4]   2> 1364106 INFO  (zkConnectionManagerCallback-2812-thread-1-processing-n:127.0.0.1:33182_solr) [n:127.0.0.1:33182_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 1364136 INFO  (zkConnectionManagerCallback-2812-thread-1-processing-n:127.0.0.1:33182_solr) [n:127.0.0.1:33182_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33182_solr
   [junit4]   2> 1364138 INFO  (zkCallback-2813-thread-9-processing-n:127.0.0.1:61851_solr) [n:127.0.0.1:61851_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1364144 INFO  (zkCallback-2811-thread-6-processing-n:127.0.0.1:33182_solr) [n:127.0.0.1:33182_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1364144 INFO  (zkConnectionManagerCallback-2812-thread-1-processing-n:127.0.0.1:33182_solr) [n:127.0.0.1:33182_solr    ] o.a.s.c.c.DefaultConnectionStrategy Reconnected to ZooKeeper
   [junit4]   2> 1364144 INFO  (zkConnectionManagerCallback-2812-thread-1-processing-n:127.0.0.1:33182_solr) [n:127.0.0.1:33182_solr    ] o.a.s.c.c.ConnectionManager zkClient Connected:true
   [junit4]   2> 1364597 ERROR (zkConnectionManagerCallback-2826-thread-1) [    ] o.a.s.c.c.DefaultConnectionStrategy Reconnect to ZooKeeper failed:java.lang.RuntimeException: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:44826/solr within 9223372036854775807 ms
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:150)
   [junit4]   2> 	at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:57)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:141)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$1.lambda$process$0(SolrZkClient.java:267)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> Caused by: java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:44826/solr within 9223372036854775807 ms
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:232)
   [junit4]   2> 	at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:147)
   [junit4]   2> 	... 9 more
   [junit4]   2> 
   [junit4]   2> 1364597 WARN  (zkConnectionManagerCallback-2826-thread-1) [    ] o.a.s.c.c.DefaultConnectionStrategy Reconnect to ZooKeeper failed
   [junit4]   2> 1364597 INFO  (zkConnectionManagerCallback-2826-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient Connected:false
   [junit4]   2> 1364600 INFO  (jetty-closer-2796-thread-1) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@4806856c{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 1364600 INFO  (jetty-closer-2796-thread-1) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1981490344
   [junit4]   2> 1364600 INFO  (jetty-closer-2796-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 1364600 INFO  (jetty-closer-2796-thread-1) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@43610cb1: rootName = solr_33182, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@40545ea6
   [junit4]   2> 1364606 INFO  (jetty-closer-2796-thread-2) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@74a58972{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 1364608 INFO  (jetty-closer-2796-thread-2) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1218508157
   [junit4]   2> 1364610 INFO  (jetty-closer-2796-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 1364610 INFO  (jetty-closer-2796-thread-1) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1f39137: rootName = solr_33182, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@17120b45
   [junit4]   2> 1364613 INFO  (jetty-closer-2796-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 1364613 INFO  (jetty-closer-2796-thread-1) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@62e2cc0b: rootName = solr_33182, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@116fb534
   [junit4]   2> 1364613 INFO  (jetty-closer-2796-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 1364613 INFO  (jetty-closer-2796-thread-2) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@33296c53: rootName = solr_61851, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@374013d2
   [junit4]   2> 1364615 INFO  (jetty-closer-2796-thread-1) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:33182_solr
   [junit4]   2> 1364616 INFO  (zkCallback-2811-thread-6-processing-n:127.0.0.1:33182_solr) [n:127.0.0.1:33182_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 1364616 INFO  (zkCallback-2813-thread-9-processing-n:127.0.0.1:61851_solr) [n:127.0.0.1:61851_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 1364616 INFO  (jetty-closer-2796-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 1364619 INFO  (jetty-closer-2796-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 1364620 INFO  (jetty-closer-2796-thread-2) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2a49d5d: rootName = solr_61851, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@17120b45
   [junit4]   2> 1364622 ERROR (Thread[Thread-3873,5,TGRP-TestSolrCloudWithSecureImpersonation]) [n:127.0.0.1:33182_solr    ] o.a.h.s.t.d.AbstractDelegationTokenSecretManager ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted
   [junit4]   2> 1364625 INFO  (jetty-closer-2796-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 1364625 INFO  (jetty-closer-2796-thread-2) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2bf7a14a: rootName = solr_61851, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@116fb534
   [junit4]   2> 1364627 INFO  (jetty-closer-2796-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@489df574{/solr,null,UNAVAILABLE}
   [junit4]   2> 1364628 INFO  (jetty-closer-2796-thread-1) [    ] o.e.j.s.session Stopped scavenging
   [junit4]   2> 1364629 INFO  (jetty-closer-2796-thread-2) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:61851_solr
   [junit4]   2> 1364631 INFO  (jetty-closer-2796-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 1364631 INFO  (coreCloseExecutor-5448-thread-1) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore [forwardingCollection_shard1_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@4977a907
   [junit4]   2> 1364632 INFO  (coreCloseExecutor-5448-thread-1) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.forwardingCollection.shard1.replica_n1, tag=1232578823
   [junit4]   2> 1364632 INFO  (zkCallback-2813-thread-9-processing-n:127.0.0.1:61851_solr) [n:127.0.0.1:61851_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 1364632 INFO  (coreCloseExecutor-5448-thread-1) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6fa3c31a: rootName = solr_61851, domain = solr.core.forwardingCollection.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.forwardingCollection.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@72fe0f82
   [junit4]   2> 1364633 INFO  (zkCallback-2813-thread-10-processing-n:127.0.0.1:61851_solr) [n:127.0.0.1:61851_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [0])
   [junit4]   2> 1364634 INFO  (zkCallback-2813-thread-3-processing-n:127.0.0.1:61851_solr) [n:127.0.0.1:61851_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [0])
   [junit4]   2> 1364643 INFO  (coreCloseExecutor-5448-thread-1) [n:127.0.0.1:61851_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.forwardingCollection.shard1.leader, tag=1232578823
   [junit4]   2> 1364645 INFO  (jetty-closer-2796-thread-2) [    ] o.a.s.c.Overseer Overseer (id=72062113543946245-127.0.0.1:61851_solr-n_0000000000) closing
   [junit4]   2> 1364645 INFO  (OverseerStateUpdate-72062113543946245-127.0.0.1:61851_solr-n_0000000000) [n:127.0.0.1:61851_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:61851_solr
   [junit4]   2> 1364646 WARN  (OverseerAutoScalingTriggerThread-72062113543946245-127.0.0.1:61851_solr-n_0000000000) [n:127.0.0.1:61851_solr    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 1364652 ERROR (Thread[Thread-3876,5,TGRP-TestSolrCloudWithSecureImpersonation]) [n:127.0.0.1:61851_solr    ] o.a.h.s.t.d.AbstractDelegationTokenSecretManager ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted
   [junit4]   2> 1364665 INFO  (jetty-closer-2796-thread-2) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@6dce18c3{/solr,null,UNAVAILABLE}
   [junit4]   2> 1364667 INFO  (jetty-closer-2796-thread-2) [    ] o.e.j.s.session Stopped scavenging
   [junit4]   2> 1364671 ERROR (SUITE-TestSolrCloudWithSecureImpersonation-seed#[CA3465D8D0D03170]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1364671 INFO  (SUITE-TestSolrCloudWithSecureImpersonation-seed#[CA3465D8D0D03170]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:44826 44826
   [junit4]   2> 1375097 INFO  (Thread-3856) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:44826 44826
   [junit4]   2> 1375098 WARN  (Thread-3856) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	4	/solr/aliases.json
   [junit4]   2> 	4	/solr/clusterprops.json
   [junit4]   2> 	3	/solr/security.json
   [junit4]   2> 	2	/solr/collections/forwardingCollection/terms/shard1
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	6	/solr/collections/forwardingCollection/state.json
   [junit4]   2> 	4	/solr/clusterstate.json
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMKeyIdRoot
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_1
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_3
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_2
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_4
   [junit4]   2> 	2	/solr/overseer_elect/election/72062113543946245-127.0.0.1:61851_solr-n_0000000000
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMSeqNumRoot
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	4	/solr/live_nodes
   [junit4]   2> 	4	/solr/collections
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot
   [junit4]   2> 	2	/solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot
   [junit4]   2> 
   [junit4]   2> 1375188 INFO  (SUITE-TestSolrCloudWithSecureImpersonation-seed#[CA3465D8D0D03170]-worker) [    ] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released
   [junit4]   2> Mar 01, 2018 7:17:39 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> 1385093 ERROR (jetty-launcher-2795-thread-1-EventThread) [n:127.0.0.1:33182_solr    ] o.a.c.f.i.CuratorFrameworkImpl Watcher exception
   [junit4]   2> java.lang.IllegalStateException: Client is not started
   [junit4]   2> 	at com.google.common.base.Preconditions.checkState(Preconditions.java:149)
   [junit4]   2> 	at org.apache.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:113)
   [junit4]   2> 	at org.apache.curator.framework.imps.CuratorFrameworkImpl.getZooKeeper(CuratorFrameworkImpl.java:477)
   [junit4]   2> 	at org.apache.curator.framework.imps.NamespaceFacade.getZooKeeper(NamespaceFacade.java:112)
   [junit4]   2> 	at org.apache.curator.framework.imps.GetDataBuilderImpl$4.call(GetDataBuilderImpl.java:302)
   [junit4]   2> 	at org.apache.curator.framework.imps.GetDataBuilderImpl$4.call(GetDataBuilderImpl.java:291)
   [junit4]   2> 	at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107)
   [junit4]   2> 	at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
   [junit4]   2> 	at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
   [junit4]   2> 	at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
   [junit4]   2> 	at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
   [junit4]   2> 	at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
   [junit4]   2> 	at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
   [junit4]   2> 	at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:531)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
   [junit4]   2> 1385096 ERROR (jetty-launcher-2795-thread-2-EventThread) [n:127.0.0.1:61851_solr    ] o.a.c.f.i.CuratorFrameworkImpl Watcher exception
   [junit4]   2> java.lang.IllegalStateException: Client is not started
   [junit4]   2> 	at com.google.common.base.Preconditions.checkState(Preconditions.java:149)
   [junit4]   2> 	at org.apache.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:113)
   [junit4]   2> 	at org.apache.curator.framework.imps.CuratorFrameworkImpl.getZooKeeper(CuratorFrameworkImpl.java:477)
   [junit4]   2> 	at org.apache.curator.framework.imps.NamespaceFacade.getZooKeeper(NamespaceFacade.java:112)
   [junit4]   2> 	at org.apache.curator.framework.imps.GetDataBuilderImpl$4.call(GetDataBuilderImpl.java:302)
   [junit4]   2> 	at org.apache.curator.framework.imps.GetDataBuilderImpl$4.call(GetDataBuilderImpl.java:291)
   [junit4]   2> 	at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107)
   [junit4]   2> 	at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
   [junit4]   2> 	at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
   [junit4]   2> 	at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
   [junit4]   2> 	at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
   [junit4]   2> 	at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
   [junit4]   2> 	at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
   [junit4]   2> 	at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:531)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506)
   [junit4]   2> 1385097 WARN  (jetty-launcher-2795-thread-2-EventThread) [n:127.0.0.1:61851_solr    ] o.a.c.ConnectionState Session expired event received
   [junit4]   2> 1385097 WARN  (jetty-launcher-2795-thread-1-EventThread) [n:127.0.0.1:33182_solr    ] o.a.c.ConnectionState Session expired event received
   [junit4]   2> 1385098 WARN  (jetty-launcher-2795-thread-2-SendThread(127.0.0.1:44826)) [n:127.0.0.1:61851_solr    ] o.a.z.ClientCnxn Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
   [junit4]   2> java.net.ConnectException: Connection refused
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1144)
   [junit4]   2> 1385098 WARN  (jetty-launcher-2795-thread-1-SendThread(127.0.0.1:44826)) [n:127.0.0.1:33182_solr    ] o.a.z.ClientCnxn Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
   [junit4]   2> java.net.ConnectException: Connection refused
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnx

[...truncated too long message...]

Provider.java:105)
   [junit4]   2>         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
   [junit4]   2>         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1412, maxMBSortInHeap=6.04148488276154, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@63094827), locale=sr, timezone=Europe/Prague
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_162 (64-bit)/cpus=3,threads=3,free=208710856,total=522190848
   [junit4]   2> NOTE: All tests run in this JVM: [AnalysisAfterCoreReloadTest, CursorPagingTest, BJQParserTest, HdfsSyncSliceTest, TestManagedStopFilterFactory, VMParamsZkACLAndCredentialsProvidersTest, TestLegacyTerms, TestRandomRequestDistribution, TestLegacyFieldCache, PingRequestHandlerTest, RequiredFieldsTest, MoveReplicaHDFSFailoverTest, SpellingQueryConverterTest, TestCoreBackup, CreateCollectionCleanupTest, TestMultiWordSynonyms, LeaderFailoverAfterPartitionTest, RAMDirectoryFactoryTest, SharedFSAutoReplicaFailoverTest, OverseerModifyCollectionTest, ZkCLITest, TestRealTimeGet, TestOmitPositions, TestCharFilters, CdcrUpdateLogTest, BlobRepositoryCloudTest, OverriddenZkACLAndCredentialsProvidersTest, PropertiesRequestHandlerTest, TestManagedSynonymGraphFilterFactory, BaseCdcrDistributedZkTest, MultiTermTest, TestCloudManagedSchema, TestInitQParser, StatelessScriptUpdateProcessorFactoryTest, CopyFieldTest, TestFunctionQuery, LeaderElectionTest, ConcurrentDeleteAndCreateCollectionTest, TestCollectionsAPIViaSolrCloudCluster, AssignBackwardCompatibilityTest, OverseerStatusTest, TestFastOutputStream, ForceLeaderTest, TestQuerySenderNoQuery, RestartWhileUpdatingTest, V2ApiIntegrationTest, TestBulkSchemaAPI, SuggesterTSTTest, TestInPlaceUpdatesStandalone, SolrCoreTest, MergeStrategyTest, RecoveryZkTest, TestImpersonationWithHadoopAuth, TestFieldSortValues, LIRRollingUpdatesTest, TestDocBasedVersionConstraints, SolrIndexSplitterTest, QueryEqualityTest, SpellCheckCollatorWithCollapseTest, TestClusterStateMutator, TestMaxScoreQueryParser, HLLSerializationTest, TestHdfsUpdateLog, CollectionsAPISolrJTest, TestEmbeddedSolrServerSchemaAPI, TestBlobHandler, SpatialFilterTest, DistributedQueryElevationComponentTest, TestBM25SimilarityFactory, TestNumericTokenStream, DistributedFacetPivotLargeTest, TestReplicationHandler, PKIAuthenticationIntegrationTest, OpenCloseCoreStressTest, TestExecutePlanAction, NodeLostTriggerTest, ParsingFieldUpdateProcessorsTest, SegmentsInfoRequestHandlerTest, TestReRankQParserPlugin, OutputWriterTest, BinaryUpdateRequestHandlerTest, JSONWriterTest, TestSchemaNameResource, TestRemoteStreaming, TestManagedSchemaThreadSafety, TestFaceting, TestSuggestSpellingConverter, TestTrie, SampleTest, BlockJoinFacetRandomTest, TestRangeQuery, TestSystemIdResolver, LeaderInitiatedRecoveryOnShardRestartTest, TestPushWriter, BasicZkTest, SSLMigrationTest, HdfsWriteToMultipleCollectionsTest, AtomicUpdateProcessorFactoryTest, SchemaVersionSpecificBehaviorTest, TestExpandComponent, TestPseudoReturnFields, HdfsChaosMonkeyNothingIsSafeTest, TestXmlQParser, NumericFieldsTest, TestLeaderElectionWithEmptyReplica, TestUtils, AlternateDirectoryTest, SolrJmxReporterCloudTest, ConnectionReuseTest, TestStressInPlaceUpdates, DirectUpdateHandlerOptimizeTest, DistributedQueryComponentOptimizationTest, TestPostingsSolrHighlighter, TestStressUserVersions, SolrPluginUtilsTest, ClassificationUpdateProcessorTest, TestClassicSimilarityFactory, TestComputePlanAction, ConvertedLegacyTest, SystemLogListenerTest, TestLegacyFieldReuse, GraphQueryTest, CdcrReplicationHandlerTest, TestExactStatsCache, TestRequestForwarding, TestUseDocValuesAsStored2, TestCryptoKeys, HttpPartitionOnCommitTest, TestTolerantSearch, PeerSyncReplicationTest, UninvertDocValuesMergePolicyTest, TestDynamicFieldCollectionResource, FieldMutatingUpdateProcessorTest, SolrIndexConfigTest, TestOverriddenPrefixQueryForCustomFieldType, SpatialHeatmapFacetsTest, RuleEngineTest, SignatureUpdateProcessorFactoryTest, MoveReplicaTest, TestConfigOverlay, SpatialRPTFieldTypeTest, TestImplicitCoreProperties, PolyFieldTest, UniqFieldsUpdateProcessorFactoryTest, CloudExitableDirectoryReaderTest, TestSubQueryTransformer, TestNonDefinedSimilarityFactory, TestSolrCloudWithSecureImpersonation]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestSolrCloudWithSecureImpersonation -Dtests.seed=CA3465D8D0D03170 -Dtests.slow=true -Dtests.locale=sr -Dtests.timezone=Europe/Prague -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J0 | TestSolrCloudWithSecureImpersonation (suite) <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 4 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation: 
   [junit4]    >    1) Thread[id=14020, name=jetty-launcher-2795-thread-1-EventThread, state=WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
   [junit4]    >    2) Thread[id=14018, name=jetty-launcher-2795-thread-2-EventThread, state=WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
   [junit4]    >    3) Thread[id=14019, name=jetty-launcher-2795-thread-1-SendThread(127.0.0.1:44826), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
   [junit4]    >         at java.lang.Thread.sleep(Native Method)
   [junit4]    >         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
   [junit4]    >    4) Thread[id=14017, name=jetty-launcher-2795-thread-2-SendThread(127.0.0.1:44826), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
   [junit4]    >         at java.lang.Thread.sleep(Native Method)
   [junit4]    >         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([CA3465D8D0D03170]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=14019, name=jetty-launcher-2795-thread-1-SendThread(127.0.0.1:44826), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
   [junit4]    >         at java.lang.Thread.sleep(Native Method)
   [junit4]    >         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
   [junit4]    >    2) Thread[id=14017, name=jetty-launcher-2795-thread-2-SendThread(127.0.0.1:44826), state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
   [junit4]    >         at java.lang.Thread.sleep(Native Method)
   [junit4]    >         at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
   [junit4]    >         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([CA3465D8D0D03170]:0)
   [junit4] Completed [287/778 (1!)] on J0 in 126.92s, 12 tests, 2 errors <<< FAILURES!

[...truncated 49386 lines...]
[repro] Jenkins log URL: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1706/consoleText

[repro] Revision: 2289c27c947f6f6a7cc96ae1839ba53e5ccc3fc1

[repro] Ant options: "-Dargs=-XX:-UseCompressedOops -XX:+UseParallelGC"
[repro] Repro line:  ant test  -Dtestcase=TestSolrCloudWithSecureImpersonation -Dtests.seed=CA3465D8D0D03170 -Dtests.slow=true -Dtests.locale=sr -Dtests.timezone=Europe/Prague -Dtests.asserts=true -Dtests.file.encoding=UTF-8

[repro] ant clean

[...truncated 6 lines...]
[repro] Test suites by module:
[repro]    solr/core
[repro]       TestSolrCloudWithSecureImpersonation
[repro] ant compile-test

[...truncated 3313 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 -Dtests.class="*.TestSolrCloudWithSecureImpersonation" -Dtests.showOutput=onerror "-Dargs=-XX:-UseCompressedOops -XX:+UseParallelGC" -Dtests.seed=CA3465D8D0D03170 -Dtests.slow=true -Dtests.locale=sr -Dtests.timezone=Europe/Prague -Dtests.asserts=true -Dtests.file.encoding=UTF-8

[...truncated 123 lines...]
[repro] Failures:
[repro]   0/5 failed: org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation
[repro] Exiting with code 0

[...truncated 70 lines...]

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 1709 - Still Unstable!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1709/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseParallelGC

9 tests failed.
FAILED:  org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testScheduledTrigger

Error Message:


Stack Trace:
java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([61F3443A36694643:F2E80C4868941D77]:0)
	at org.junit.Assert.fail(Assert.java:92)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNotNull(Assert.java:526)
	at org.junit.Assert.assertNotNull(Assert.java:537)
	at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testScheduledTrigger(TriggerIntegrationTest.java:1683)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testScheduledTrigger

Error Message:


Stack Trace:
java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([61F3443A36694643:F2E80C4868941D77]:0)
	at org.junit.Assert.fail(Assert.java:92)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNotNull(Assert.java:526)
	at org.junit.Assert.assertNotNull(Assert.java:537)
	at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testScheduledTrigger(TriggerIntegrationTest.java:1683)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testScheduledTrigger

Error Message:


Stack Trace:
java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([61F3443A36694643:F2E80C4868941D77]:0)
	at org.junit.Assert.fail(Assert.java:92)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNotNull(Assert.java:526)
	at org.junit.Assert.assertNotNull(Assert.java:537)
	at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testScheduledTrigger(TriggerIntegrationTest.java:1683)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testScheduledTrigger

Error Message:


Stack Trace:
java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([61F3443A36694643:F2E80C4868941D77]:0)
	at org.junit.Assert.fail(Assert.java:92)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNotNull(Assert.java:526)
	at org.junit.Assert.assertNotNull(Assert.java:537)
	at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testScheduledTrigger(TriggerIntegrationTest.java:1683)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testScheduledTrigger

Error Message:


Stack Trace:
java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([61F3443A36694643:F2E80C4868941D77]:0)
	at org.junit.Assert.fail(Assert.java:92)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNotNull(Assert.java:526)
	at org.junit.Assert.assertNotNull(Assert.java:537)
	at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testScheduledTrigger(TriggerIntegrationTest.java:1683)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testScheduledTrigger

Error Message:


Stack Trace:
java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([61F3443A36694643:F2E80C4868941D77]:0)
	at org.junit.Assert.fail(Assert.java:92)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNotNull(Assert.java:526)
	at org.junit.Assert.assertNotNull(Assert.java:537)
	at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testScheduledTrigger(TriggerIntegrationTest.java:1683)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testEventQueue

Error Message:
action wasn't interrupted

Stack Trace:
java.lang.AssertionError: action wasn't interrupted
	at __randomizedtesting.SeedInfo.seed([61F3443A36694643:A84606943F0E80B6]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testEventQueue(TriggerIntegrationTest.java:729)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testEventQueue

Error Message:
action wasn't interrupted

Stack Trace:
java.lang.AssertionError: action wasn't interrupted
	at __randomizedtesting.SeedInfo.seed([61F3443A36694643:A84606943F0E80B6]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testEventQueue(TriggerIntegrationTest.java:729)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testEventQueue

Error Message:
action wasn't interrupted

Stack Trace:
java.lang.AssertionError: action wasn't interrupted
	at __randomizedtesting.SeedInfo.seed([61F3443A36694643:A84606943F0E80B6]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testEventQueue(TriggerIntegrationTest.java:729)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 14069 lines...]
   [junit4] Suite: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_61F3443A36694643-001/init-core-data-001
   [junit4]   2> 3589982 WARN  (SUITE-TriggerIntegrationTest-seed#[61F3443A36694643]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=119 numCloses=119
   [junit4]   2> 3589982 INFO  (SUITE-TriggerIntegrationTest-seed#[61F3443A36694643]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 3589987 INFO  (SUITE-TriggerIntegrationTest-seed#[61F3443A36694643]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 3589987 INFO  (SUITE-TriggerIntegrationTest-seed#[61F3443A36694643]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 3589987 INFO  (SUITE-TriggerIntegrationTest-seed#[61F3443A36694643]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_61F3443A36694643-001/tempDir-001
   [junit4]   2> 3589987 INFO  (SUITE-TriggerIntegrationTest-seed#[61F3443A36694643]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 3589988 INFO  (Thread-5914) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 3589988 INFO  (Thread-5914) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 3589989 ERROR (Thread-5914) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 3590088 INFO  (SUITE-TriggerIntegrationTest-seed#[61F3443A36694643]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:60193
   [junit4]   2> 3590090 INFO  (zkConnectionManagerCallback-7299-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3590095 INFO  (jetty-launcher-7296-thread-1) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T22:27:37+01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 3590095 INFO  (jetty-launcher-7296-thread-2) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T22:27:37+01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 3590096 INFO  (jetty-launcher-7296-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 3590097 INFO  (jetty-launcher-7296-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 3590097 INFO  (jetty-launcher-7296-thread-2) [    ] o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 3590097 INFO  (jetty-launcher-7296-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 3590097 INFO  (jetty-launcher-7296-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 3590097 INFO  (jetty-launcher-7296-thread-1) [    ] o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 3590097 INFO  (jetty-launcher-7296-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@282535d4{/solr,null,AVAILABLE}
   [junit4]   2> 3590097 INFO  (jetty-launcher-7296-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3875d3b7{/solr,null,AVAILABLE}
   [junit4]   2> 3590097 INFO  (jetty-launcher-7296-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@3507841{HTTP/1.1,[http/1.1]}{127.0.0.1:52358}
   [junit4]   2> 3590098 INFO  (jetty-launcher-7296-thread-1) [    ] o.e.j.s.Server Started @3593200ms
   [junit4]   2> 3590098 INFO  (jetty-launcher-7296-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=52358}
   [junit4]   2> 3590098 INFO  (jetty-launcher-7296-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@d008f27{HTTP/1.1,[http/1.1]}{127.0.0.1:37296}
   [junit4]   2> 3590098 INFO  (jetty-launcher-7296-thread-2) [    ] o.e.j.s.Server Started @3593201ms
   [junit4]   2> 3590098 INFO  (jetty-launcher-7296-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37296}
   [junit4]   2> 3590098 ERROR (jetty-launcher-7296-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 3590098 ERROR (jetty-launcher-7296-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 3590098 INFO  (jetty-launcher-7296-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 3590099 INFO  (jetty-launcher-7296-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 3590099 INFO  (jetty-launcher-7296-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 3590099 INFO  (jetty-launcher-7296-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 3590099 INFO  (jetty-launcher-7296-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 3590099 INFO  (jetty-launcher-7296-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 3590099 INFO  (jetty-launcher-7296-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-03-03T02:26:18.284Z
   [junit4]   2> 3590099 INFO  (jetty-launcher-7296-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-03-03T02:26:18.284Z
   [junit4]   2> 3590100 INFO  (zkConnectionManagerCallback-7303-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3590101 INFO  (zkConnectionManagerCallback-7301-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3590101 INFO  (jetty-launcher-7296-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 3590101 INFO  (jetty-launcher-7296-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 3590108 INFO  (jetty-launcher-7296-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60193/solr
   [junit4]   2> 3590108 INFO  (jetty-launcher-7296-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60193/solr
   [junit4]   2> 3590109 INFO  (zkConnectionManagerCallback-7309-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3590110 INFO  (zkConnectionManagerCallback-7311-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3590112 INFO  (zkConnectionManagerCallback-7313-thread-1-processing-n:127.0.0.1:52358_solr) [n:127.0.0.1:52358_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3590113 INFO  (zkConnectionManagerCallback-7315-thread-1-processing-n:127.0.0.1:37296_solr) [n:127.0.0.1:37296_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3590256 INFO  (jetty-launcher-7296-thread-2) [n:127.0.0.1:37296_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3590257 INFO  (jetty-launcher-7296-thread-2) [n:127.0.0.1:37296_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37296_solr
   [junit4]   2> 3590258 INFO  (jetty-launcher-7296-thread-2) [n:127.0.0.1:37296_solr    ] o.a.s.c.Overseer Overseer (id=72069465497600006-127.0.0.1:37296_solr-n_0000000000) starting
   [junit4]   2> 3590262 INFO  (jetty-launcher-7296-thread-1) [n:127.0.0.1:52358_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3590263 INFO  (jetty-launcher-7296-thread-1) [n:127.0.0.1:52358_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52358_solr
   [junit4]   2> 3590265 INFO  (zkCallback-7314-thread-1-processing-n:127.0.0.1:37296_solr) [n:127.0.0.1:37296_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 3590266 INFO  (zkCallback-7312-thread-1-processing-n:127.0.0.1:52358_solr) [n:127.0.0.1:52358_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 3590270 DEBUG (OverseerAutoScalingTriggerThread-72069465497600006-127.0.0.1:37296_solr-n_0000000000) [n:127.0.0.1:37296_solr    ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
   [junit4]   2> 3590271 DEBUG (OverseerAutoScalingTriggerThread-72069465497600006-127.0.0.1:37296_solr-n_0000000000) [n:127.0.0.1:37296_solr    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 3590271 INFO  (jetty-launcher-7296-thread-2) [n:127.0.0.1:37296_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37296_solr
   [junit4]   2> 3590271 DEBUG (OverseerAutoScalingTriggerThread-72069465497600006-127.0.0.1:37296_solr-n_0000000000) [n:127.0.0.1:37296_solr    ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:52358_solr]
   [junit4]   2> 3590271 DEBUG (OverseerAutoScalingTriggerThread-72069465497600006-127.0.0.1:37296_solr-n_0000000000) [n:127.0.0.1:37296_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
   [junit4]   2> 3590271 DEBUG (OverseerAutoScalingTriggerThread-72069465497600006-127.0.0.1:37296_solr-n_0000000000) [n:127.0.0.1:37296_solr    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 3590273 INFO  (zkCallback-7314-thread-1-processing-n:127.0.0.1:37296_solr) [n:127.0.0.1:37296_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 3590274 DEBUG (OverseerAutoScalingTriggerThread-72069465497600006-127.0.0.1:37296_solr-n_0000000000) [n:127.0.0.1:37296_solr    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 3590275 DEBUG (OverseerAutoScalingTriggerThread-72069465497600006-127.0.0.1:37296_solr-n_0000000000) [n:127.0.0.1:37296_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 3590275 DEBUG (ScheduledTrigger-11159-thread-1) [n:127.0.0.1:37296_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3590276 INFO  (zkCallback-7312-thread-1-processing-n:127.0.0.1:52358_solr) [n:127.0.0.1:52358_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 3590424 INFO  (jetty-launcher-7296-thread-1) [n:127.0.0.1:52358_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52358.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@41a18e4b
   [junit4]   2> 3590431 INFO  (jetty-launcher-7296-thread-1) [n:127.0.0.1:52358_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52358.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@41a18e4b
   [junit4]   2> 3590431 INFO  (jetty-launcher-7296-thread-1) [n:127.0.0.1:52358_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52358.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@41a18e4b
   [junit4]   2> 3590432 INFO  (jetty-launcher-7296-thread-1) [n:127.0.0.1:52358_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_61F3443A36694643-001/tempDir-001/node1/.
   [junit4]   2> 3590465 INFO  (jetty-launcher-7296-thread-2) [n:127.0.0.1:37296_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37296.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@41a18e4b
   [junit4]   2> 3590474 INFO  (jetty-launcher-7296-thread-2) [n:127.0.0.1:37296_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37296.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@41a18e4b
   [junit4]   2> 3590474 INFO  (jetty-launcher-7296-thread-2) [n:127.0.0.1:37296_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37296.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@41a18e4b
   [junit4]   2> 3590476 INFO  (jetty-launcher-7296-thread-2) [n:127.0.0.1:37296_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_61F3443A36694643-001/tempDir-001/node2/.
   [junit4]   2> 3590512 INFO  (zkConnectionManagerCallback-7321-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3590516 INFO  (zkConnectionManagerCallback-7325-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3590518 INFO  (SUITE-TriggerIntegrationTest-seed#[61F3443A36694643]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 3590520 INFO  (SUITE-TriggerIntegrationTest-seed#[61F3443A36694643]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:60193/solr ready
   [junit4]   2> 3590545 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testListeners
   [junit4]   2> 3590563 INFO  (qtp817857837-30319) [n:127.0.0.1:37296_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :overseerstatus with params action=OVERSEERSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 3590577 INFO  (qtp817857837-30319) [n:127.0.0.1:37296_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=OVERSEERSTATUS&wt=javabin&version=2} status=0 QTime=13
   [junit4]   2> 3590579 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.a.s.c.a.ScheduledTriggers -- removing trigger: .auto_add_replicas
   [junit4]   2> 3590582 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 2
   [junit4]   2> 3590582 DEBUG (zkCallback-7314-thread-1-processing-n:127.0.0.1:37296_solr) [n:127.0.0.1:37296_solr    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
   [junit4]   2> 3590582 DEBUG (OverseerAutoScalingTriggerThread-72069465497600006-127.0.0.1:37296_solr-n_0000000000) [n:127.0.0.1:37296_solr    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
   [junit4]   2> 3590582 DEBUG (OverseerAutoScalingTriggerThread-72069465497600006-127.0.0.1:37296_solr-n_0000000000) [n:127.0.0.1:37296_solr    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
   [junit4]   2> 3590587 DEBUG (OverseerAutoScalingTriggerThread-72069465497600006-127.0.0.1:37296_solr-n_0000000000) [n:127.0.0.1:37296_solr    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 3590589 DEBUG (OverseerAutoScalingTriggerThread-72069465497600006-127.0.0.1:37296_solr-n_0000000000) [n:127.0.0.1:37296_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
   [junit4]   2> 3590589 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 3590597 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@d008f27{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 3590598 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=240266750
   [junit4]   2> 3590598 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 3590598 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7ad9d009: rootName = solr_37296, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@7b06b4e1
   [junit4]   2> 3590606 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 3590606 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@400adeaf: rootName = solr_37296, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@21ce5a73
   [junit4]   2> 3590610 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 3590610 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1fb83206: rootName = solr_37296, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@36b2616c
   [junit4]   2> 3590612 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:37296_solr
   [junit4]   2> 3590615 INFO  (zkCallback-7312-thread-1-processing-n:127.0.0.1:52358_solr) [n:127.0.0.1:52358_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 3590624 INFO  (zkCallback-7324-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 3590624 INFO  (zkCallback-7324-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 3590624 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 3590624 INFO  (zkCallback-7314-thread-1-processing-n:127.0.0.1:37296_solr) [n:127.0.0.1:37296_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 3590625 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 3590625 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.a.s.c.Overseer Overseer (id=72069465497600006-127.0.0.1:37296_solr-n_0000000000) closing
   [junit4]   2> 3590625 INFO  (OverseerStateUpdate-72069465497600006-127.0.0.1:37296_solr-n_0000000000) [n:127.0.0.1:37296_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:37296_solr
   [junit4]   2> 3590626 WARN  (OverseerAutoScalingTriggerThread-72069465497600006-127.0.0.1:37296_solr-n_0000000000) [n:127.0.0.1:37296_solr    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 3590626 DEBUG (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
   [junit4]   2> 3590628 INFO  (zkCallback-7312-thread-1-processing-n:127.0.0.1:52358_solr) [n:127.0.0.1:52358_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:52358_solr
   [junit4]   2> 3590628 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@282535d4{/solr,null,UNAVAILABLE}
   [junit4]   2> 3590629 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.e.j.s.session Stopped scavenging
   [junit4]   2> 3590632 INFO  (zkCallback-7312-thread-1-processing-n:127.0.0.1:52358_solr) [n:127.0.0.1:52358_solr    ] o.a.s.c.Overseer Overseer (id=72069465497600005-127.0.0.1:52358_solr-n_0000000001) starting
   [junit4]   2> 3590643 DEBUG (OverseerAutoScalingTriggerThread-72069465497600005-127.0.0.1:52358_solr-n_0000000001) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
   [junit4]   2> 3590644 DEBUG (OverseerAutoScalingTriggerThread-72069465497600005-127.0.0.1:52358_solr-n_0000000001) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
   [junit4]   2> 3590645 DEBUG (OverseerAutoScalingTriggerThread-72069465497600005-127.0.0.1:52358_solr-n_0000000001) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:52358_solr]
   [junit4]   2> 3590645 DEBUG (OverseerAutoScalingTriggerThread-72069465497600005-127.0.0.1:52358_solr-n_0000000001) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion -1
   [junit4]   2> 3590645 DEBUG (OverseerAutoScalingTriggerThread-72069465497600005-127.0.0.1:52358_solr-n_0000000001) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
   [junit4]   2> 3590649 DEBUG (OverseerAutoScalingTriggerThread-72069465497600005-127.0.0.1:52358_solr-n_0000000001) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 3590649 DEBUG (OverseerAutoScalingTriggerThread-72069465497600005-127.0.0.1:52358_solr-n_0000000001) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
   [junit4]   2> 3590650 DEBUG (ScheduledTrigger-11168-thread-1) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 3591652 DEBUG (ScheduledTrigger-11168-thread-1) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 3592670 DEBUG (ScheduledTrigger-11168-thread-1) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 3593672 DEBUG (ScheduledTrigger-11168-thread-1) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 3594673 DEBUG (ScheduledTrigger-11168-thread-1) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
   [junit4]   2> 3595632 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T22:27:37+01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 3595633 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 3595633 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 3595633 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 3595633 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5a92d6a2{/solr,null,AVAILABLE}
   [junit4]   2> 3595633 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@4bcfdf4b{HTTP/1.1,[http/1.1]}{127.0.0.1:40851}
   [junit4]   2> 3595633 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.e.j.s.Server Started @3598737ms
   [junit4]   2> 3595633 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40851}
   [junit4]   2> 3595634 ERROR (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 3595635 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 3595635 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 3595635 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 3595635 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-03-03T02:26:23.820Z
   [junit4]   2> 3595637 INFO  (zkConnectionManagerCallback-7328-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3595637 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 3595642 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1000acc0a230009, likely client has closed socket
   [junit4]   2> 3595644 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60193/solr
   [junit4]   2> 3595646 INFO  (zkConnectionManagerCallback-7332-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3595647 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1000acc0a23000a, likely client has closed socket
   [junit4]   2> 3595649 INFO  (zkConnectionManagerCallback-7334-thread-1-processing-n:127.0.0.1:40851_solr) [n:127.0.0.1:40851_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3595659 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [n:127.0.0.1:40851_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 3595663 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [n:127.0.0.1:40851_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3595665 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [n:127.0.0.1:40851_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40851_solr
   [junit4]   2> 3595666 INFO  (zkCallback-7312-thread-1-processing-n:127.0.0.1:52358_solr) [n:127.0.0.1:52358_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 3595667 INFO  (zkCallback-7333-thread-1-processing-n:127.0.0.1:40851_solr) [n:127.0.0.1:40851_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 3595667 INFO  (zkCallback-7324-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 3595667 INFO  (zkCallback-7324-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 3595674 DEBUG (ScheduledTrigger-11168-thread-1) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 3595729 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [n:127.0.0.1:40851_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40851.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@41a18e4b
   [junit4]   2> 3595735 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [n:127.0.0.1:40851_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40851.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@41a18e4b
   [junit4]   2> 3595735 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [n:127.0.0.1:40851_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40851.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@41a18e4b
   [junit4]   2> 3595736 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [n:127.0.0.1:40851_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_61F3443A36694643-001/tempDir-001/node3/.
   [junit4]   2> 3595764 INFO  (qtp827251383-30394) [n:127.0.0.1:40851_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
   [junit4]   2> 3595767 INFO  (qtp1540941543-30317) [n:127.0.0.1:52358_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=1
   [junit4]   2> 3595767 DEBUG (qtp827251383-30390) [n:127.0.0.1:40851_solr    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 3595768 INFO  (qtp827251383-30390) [n:127.0.0.1:40851_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=6
   [junit4]   2> 3595768 DEBUG (zkCallback-7312-thread-1-processing-n:127.0.0.1:52358_solr) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
   [junit4]   2> 3595769 DEBUG (zkCallback-7312-thread-1-processing-n:127.0.0.1:52358_solr) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:40851_solr, 127.0.0.1:52358_solr]
   [junit4]   2> 3595769 DEBUG (zkCallback-7312-thread-1-processing-n:127.0.0.1:52358_solr) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerL instantiated with properties: {event=nodeAdded, waitFor=3, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestTriggerAction}, {name=test1, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestDummyAction}], enabled=true}
   [junit4]   2> 3595769 DEBUG (zkCallback-7312-thread-1-processing-n:127.0.0.1:52358_solr) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:40851_solr, 127.0.0.1:52358_solr]
   [junit4]   2> 3595769 DEBUG (OverseerAutoScalingTriggerThread-72069465497600005-127.0.0.1:52358_solr-n_0000000001) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
   [junit4]   2> 3595771 INFO  (OverseerAutoScalingTriggerThread-72069465497600005-127.0.0.1:52358_solr-n_0000000001) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
   [junit4]   2> 3595773 DEBUG (ScheduledTrigger-11168-thread-2) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerL
   [junit4]   2> 3595773 DEBUG (ScheduledTrigger-11168-thread-2) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
   [junit4]   2> 3595774 DEBUG (OverseerAutoScalingTriggerThread-72069465497600005-127.0.0.1:52358_solr-n_0000000001) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
   [junit4]   2> 3595775 INFO  (qtp827251383-30394) [n:127.0.0.1:40851_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
   [junit4]   2> 3595776 INFO  (qtp1540941543-30320) [n:127.0.0.1:52358_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
   [junit4]   2> 3595776 DEBUG (qtp827251383-30392) [n:127.0.0.1:40851_solr    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 3595777 INFO  (qtp827251383-30392) [n:127.0.0.1:40851_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=3
   [junit4]   2> 3595777 DEBUG (zkCallback-7312-thread-1-processing-n:127.0.0.1:52358_solr) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
   [junit4]   2> 3595778 DEBUG (zkCallback-7312-thread-1-processing-n:127.0.0.1:52358_solr) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:40851_solr, 127.0.0.1:52358_solr]
   [junit4]   2> 3595778 DEBUG (zkCallback-7312-thread-1-processing-n:127.0.0.1:52358_solr) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerL instantiated with properties: {event=nodeAdded, waitFor=3, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestTriggerAction}, {name=test1, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestDummyAction}], enabled=true}
   [junit4]   2> 3595778 DEBUG (zkCallback-7312-thread-1-processing-n:127.0.0.1:52358_solr) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:40851_solr, 127.0.0.1:52358_solr]
   [junit4]   2> 3595778 DEBUG (OverseerAutoScalingTriggerThread-72069465497600005-127.0.0.1:52358_solr-n_0000000001) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
   [junit4]   2> 3595779 INFO  (qtp827251383-30392) [n:127.0.0.1:40851_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
   [junit4]   2> 3595779 INFO  (qtp1540941543-30326) [n:127.0.0.1:52358_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
   [junit4]   2> 3595779 DEBUG (OverseerAutoScalingTriggerThread-72069465497600005-127.0.0.1:52358_solr-n_0000000001) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
   [junit4]   2> 3595780 DEBUG (qtp827251383-30395) [n:127.0.0.1:40851_solr    ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
   [junit4]   2> 3595780 INFO  (qtp827251383-30395) [n:127.0.0.1:40851_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=3
   [junit4]   2> 3595781 DEBUG (zkCallback-7312-thread-1-processing-n:127.0.0.1:52358_solr) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
   [junit4]   2> 3595781 DEBUG (zkCallback-7312-thread-1-processing-n:127.0.0.1:52358_solr) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:40851_solr, 127.0.0.1:52358_solr]
   [junit4]   2> 3595781 DEBUG (zkCallback-7312-thread-1-processing-n:127.0.0.1:52358_solr) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerL instantiated with properties: {event=nodeAdded, waitFor=3, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestTriggerAction}, {name=test1, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestDummyAction}], enabled=true}
   [junit4]   2> 3595781 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T22:27:37+01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 3595781 DEBUG (zkCallback-7312-thread-1-processing-n:127.0.0.1:52358_solr) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:40851_solr, 127.0.0.1:52358_solr]
   [junit4]   2> 3595782 DEBUG (OverseerAutoScalingTriggerThread-72069465497600005-127.0.0.1:52358_solr-n_0000000001) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
   [junit4]   2> 3595782 DEBUG (OverseerAutoScalingTriggerThread-72069465497600005-127.0.0.1:52358_solr-n_0000000001) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
   [junit4]   2> 3595782 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 3595782 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 3595783 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 3595783 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@34c07bd{/solr,null,AVAILABLE}
   [junit4]   2> 3595783 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@5fbf6ccd{HTTP/1.1,[http/1.1]}{127.0.0.1:43088}
   [junit4]   2> 3595783 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.e.j.s.Server Started @3598887ms
   [junit4]   2> 3595783 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=43088}
   [junit4]   2> 3595783 ERROR (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 3595784 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 3595784 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 3595784 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 3595784 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-03-03T02:26:23.969Z
   [junit4]   2> 3595786 INFO  (zkConnectionManagerCallback-7338-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3595787 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 3595793 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60193/solr
   [junit4]   2> 3595794 INFO  (zkConnectionManagerCallback-7342-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3595797 INFO  (zkConnectionManagerCallback-7344-thread-1-processing-n:127.0.0.1:43088_solr) [n:127.0.0.1:43088_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3595808 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [n:127.0.0.1:43088_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 3595810 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [n:127.0.0.1:43088_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3595813 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [n:127.0.0.1:43088_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43088_solr
   [junit4]   2> 3595815 INFO  (zkCallback-7312-thread-1-processing-n:127.0.0.1:52358_solr) [n:127.0.0.1:52358_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 3595815 INFO  (zkCallback-7324-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 3595815 INFO  (zkCallback-7324-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 3595815 INFO  (zkCallback-7333-thread-1-processing-n:127.0.0.1:40851_solr) [n:127.0.0.1:40851_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 3595816 INFO  (zkCallback-7343-thread-1-processing-n:127.0.0.1:43088_solr) [n:127.0.0.1:43088_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 3595875 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [n:127.0.0.1:43088_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43088.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@41a18e4b
   [junit4]   2> 3595881 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [n:127.0.0.1:43088_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43088.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@41a18e4b
   [junit4]   2> 3595881 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [n:127.0.0.1:43088_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43088.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@41a18e4b
   [junit4]   2> 3595882 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [n:127.0.0.1:43088_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_61F3443A36694643-001/tempDir-001/node4/.
   [junit4]   2> 3596675 DEBUG (ScheduledTrigger-11168-thread-1) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 3596775 DEBUG (ScheduledTrigger-11168-thread-3) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerL
   [junit4]   2> 3596775 DEBUG (ScheduledTrigger-11168-thread-3) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3596775 DEBUG (ScheduledTrigger-11168-thread-3) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:43088_solr at time 181150887964160
   [junit4]   2> 3597677 DEBUG (ScheduledTrigger-11168-thread-4) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 3597777 DEBUG (ScheduledTrigger-11168-thread-2) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerL
   [junit4]   2> 3597777 DEBUG (ScheduledTrigger-11168-thread-2) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3598678 DEBUG (ScheduledTrigger-11168-thread-1) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 3598777 DEBUG (ScheduledTrigger-11168-thread-3) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerL
   [junit4]   2> 3598777 DEBUG (ScheduledTrigger-11168-thread-3) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3599679 DEBUG (ScheduledTrigger-11168-thread-4) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
   [junit4]   2> 3599777 DEBUG (ScheduledTrigger-11168-thread-2) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerL
   [junit4]   2> 3599777 DEBUG (ScheduledTrigger-11168-thread-2) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
   [junit4]   2> 3599777 DEBUG (ScheduledTrigger-11168-thread-2) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerL firing registered processor for nodes: [127.0.0.1:43088_solr] added at times [181150887964160], now=181153890231217
   [junit4]   2> 3599777 DEBUG (ScheduledTrigger-11168-thread-2) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
   [junit4]   2>   "id":"a4c17a53ee00T8d4qyeg2s8p1ehwbsydkuilz6",
   [junit4]   2>   "source":"node_added_triggerL",
   [junit4]   2>   "eventTime":181150887964160,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[181150887964160],
   [junit4]   2>     "nodeNames":["127.0.0.1:43088_solr"]}}
   [junit4]   2> 3599778 DEBUG (ScheduledTrigger-11168-thread-2) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_triggerL]
   [junit4]   2> 3599779 INFO  (ScheduledTrigger-11168-thread-2) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
   [junit4]   2> 3599780 DEBUG (AutoscalingActionExecutor-11169-thread-1-processing-n:127.0.0.1:52358_solr) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
   [junit4]   2>   "id":"a4c17a53ee00T8d4qyeg2s8p1ehwbsydkuilz6",
   [junit4]   2>   "source":"node_added_triggerL",
   [junit4]   2>   "eventTime":181150887964160,
   [junit4]   2>   "eventType":"NODEADDED",
   [junit4]   2>   "properties":{
   [junit4]   2>     "eventTimes":[181150887964160],
   [junit4]   2>     "_enqueue_time_":181153890941172,
   [junit4]   2>     "nodeNames":["127.0.0.1:43088_solr"]}}
   [junit4]   2> 3599782 DEBUG (AutoscalingActionExecutor-11169-thread-1-processing-n:127.0.0.1:52358_solr) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 5000ms
   [junit4]   2> 3599782 DEBUG (AutoscalingActionExecutor-11169-thread-1-processing-n:127.0.0.1:52358_solr) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_triggerL after 5000ms
   [junit4]   2> 3599782 DEBUG (AutoscalingActionExecutor-11169-thread-1-processing-n:127.0.0.1:52358_solr) [n:127.0.0.1:52358_solr    ] o.a.s.c.a.ScheduledTriggers -- processing took 2 ms for event id=a4c17a53ee00T8d4qyeg2s8p1ehwbsydkuilz6
   [junit4]   2> 3601783 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T22:27:37+01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 3601784 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 3601784 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 3601784 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.e.j.s.session Scavenging every 600000ms
   [junit4]   2> 3601785 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4b19b953{/solr,null,AVAILABLE}
   [junit4]   2> 3601793 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@2a1883d6{HTTP/1.1,[http/1.1]}{127.0.0.1:62020}
   [junit4]   2> 3601793 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.e.j.s.Server Started @3604898ms
   [junit4]   2> 3601793 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=62020}
   [junit4]   2> 3601794 ERROR (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 3601794 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 3601794 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 3601794 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 3601794 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-03-03T02:26:29.979Z
   [junit4]   2> 3601796 INFO  (zkConnectionManagerCallback-7348-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3601797 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 3601803 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60193/solr
   [junit4]   2> 3601805 INFO  (zkConnectionManagerCallback-7352-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3601809 INFO  (zkConnectionManagerCallback-7354-thread-1-processing-n:127.0.0.1:62020_solr) [n:127.0.0.1:62020_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3601820 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [n:127.0.0.1:62020_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 3601823 INFO  (TEST-TriggerIntegrationTest.testListeners-seed#[61F3443A36694643]) [n:127.0.0.1:62020_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4] 

[...truncated too long message...]

om ZooKeeper... (1) -> (0)
   [junit4]   2> 711589 INFO  (jetty-closer-574-thread-4) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 711592 INFO  (zkCallback-696-thread-3-processing-n:127.0.0.1:43959_solr) [n:127.0.0.1:43959_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:43959_solr
   [junit4]   2> 712993 WARN  (zkCallback-707-thread-1-processing-n:127.0.0.1:52422_solr) [n:127.0.0.1:52422_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 712995 INFO  (jetty-closer-574-thread-5) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@31d04927{/solr,null,UNAVAILABLE}
   [junit4]   2> 712995 INFO  (jetty-closer-574-thread-5) [    ] o.e.j.s.session Stopped scavenging
   [junit4]   2> 713068 WARN  (zkCallback-671-thread-1-processing-n:127.0.0.1:38748_solr) [n:127.0.0.1:38748_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 713069 INFO  (jetty-closer-574-thread-2) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1e8967c6{/solr,null,UNAVAILABLE}
   [junit4]   2> 713069 INFO  (jetty-closer-574-thread-2) [    ] o.e.j.s.session Stopped scavenging
   [junit4]   2> 713086 WARN  (zkCallback-661-thread-3-processing-n:127.0.0.1:33183_solr) [n:127.0.0.1:33183_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 713087 INFO  (jetty-closer-574-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@24b511f0{/solr,null,UNAVAILABLE}
   [junit4]   2> 713087 INFO  (jetty-closer-574-thread-1) [    ] o.e.j.s.session Stopped scavenging
   [junit4]   2> 713095 WARN  (zkCallback-696-thread-2-processing-n:127.0.0.1:43959_solr) [n:127.0.0.1:43959_solr    ] o.a.s.c.ZkController Unable to register nodeLost path for 127.0.0.1:43959_solr
   [junit4]   2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /autoscaling/nodeLost/127.0.0.1:43959_solr
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:786)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.lambda$create$7(SolrZkClient.java:365)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:365)
   [junit4]   2> 	at org.apache.solr.cloud.ZkController.lambda$registerLiveNodesListener$2(ZkController.java:885)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader.lambda$refreshLiveNodes$1(ZkStateReader.java:724)
   [junit4]   2> 	at java.util.concurrent.ConcurrentHashMap$KeySetView.forEach(ConcurrentHashMap.java:4649)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader.refreshLiveNodes(ZkStateReader.java:723)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader.access$1000(ZkStateReader.java:72)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader$LiveNodeWatcher.refreshAndWatch(ZkStateReader.java:1110)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader$LiveNodeWatcher.process(ZkStateReader.java:1105)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$1.lambda$process$1(SolrZkClient.java:269)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 713096 INFO  (jetty-closer-574-thread-4) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1519ee27{/solr,null,UNAVAILABLE}
   [junit4]   2> 713097 INFO  (jetty-closer-574-thread-4) [    ] o.e.j.s.session Stopped scavenging
   [junit4]   2> 713100 ERROR (SUITE-TriggerIntegrationTest-seed#[61F3443A36694643]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 713101 INFO  (SUITE-TriggerIntegrationTest-seed#[61F3443A36694643]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:58116 58116
   [junit4]   2> 718548 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxnFactory Ignoring unexpected runtime exception
   [junit4]   2> java.nio.channels.CancelledKeyException
   [junit4]   2> 	at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
   [junit4]   2> 	at sun.nio.ch.SelectionKeyImpl.readyOps(SelectionKeyImpl.java:87)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:205)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 718548 INFO  (Thread-522) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:58116 58116
   [junit4]   2> 718550 WARN  (Thread-522) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	19	/solr/clusterprops.json
   [junit4]   2> 	19	/solr/aliases.json
   [junit4]   2> 	14	/solr/collections/testScheduledTrigger/terms/shard1
   [junit4]   2> 	12	/solr/security.json
   [junit4]   2> 	3	/solr/configs/testScheduledTrigger.AUTOCREATED/managed-schema
   [junit4]   2> 	3	/solr/configs/testScheduledTrigger.AUTOCREATED
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	22	/solr/collections/testScheduledTrigger/state.json
   [junit4]   2> 	19	/solr/clusterstate.json
   [junit4]   2> 	8	/solr/autoscaling.json
   [junit4]   2> 	2	/solr/overseer_elect/election/72069613317718045-127.0.0.1:38748_solr-n_0000000008
   [junit4]   2> 	2	/solr/collections/testScheduledTrigger/leader_elect/shard1/election/72069613317718042-core_node6-n_0000000000
   [junit4]   2> 	2	/solr/overseer_elect/election/72069613317718042-127.0.0.1:33183_solr-n_0000000007
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	72	/solr/live_nodes
   [junit4]   2> 	28	/solr/collections
   [junit4]   2> 	8	/solr/autoscaling/events/.auto_add_replicas
   [junit4]   2> 	8	/solr/overseer/queue
   [junit4]   2> 	8	/solr/overseer/collection-queue-work
   [junit4]   2> 	8	/solr/overseer/queue-work
   [junit4]   2> 
   [junit4]   2> NOTE: leaving temporary files on disk at: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_61F3443A36694643-003
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1501, maxMBSortInHeap=6.540016856653221, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@20735270), locale=bg, timezone=Antarctica/Macquarie
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_162 (64-bit)/cpus=3,threads=1,free=120720560,total=291504128
   [junit4]   2> NOTE: All tests run in this JVM: [TriggerIntegrationTest, TriggerIntegrationTest, TriggerIntegrationTest]
   [junit4] Completed [5/5 (5!)] on J1 in 132.23s, 15 tests, 2 failures, 9 skipped <<< FAILURES!

[...truncated 17 lines...]
BUILD FAILED
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/common-build.xml:1561: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/common-build.xml:1085: There were test failures: 5 suites, 75 tests, 8 failures, 9 ignored (9 assumptions) [seed: 61F3443A36694643]

Total time: 12 minutes 3 seconds

[repro] Setting last failure code to 256

[repro] Failures:
[repro]   5/5 failed: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest
[repro] Exiting with code 256
+ mv lucene/build lucene/build.repro
+ mv solr/build solr/build.repro
+ mv lucene/build.orig lucene/build
+ mv solr/build.orig solr/build
Archiving artifacts
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
<Git Blamer> Using GitBlamer to create author and commit information for all warnings.
<Git Blamer> GIT_COMMIT=b470a63690cc4a0edabc41f5d65e17994442a4a9, workspace=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris
[WARNINGS] Computing warning deltas based on reference build #1708
Recording test results
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/export/home/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 1708 - Still Unstable!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1708/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  org.apache.solr.cloud.AliasIntegrationTest.testModifyMetadataCAR

Error Message:
Error from server at http://127.0.0.1:43196/solr: Can't modify non-existent alias testModifyMetadataCAR

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:43196/solr: Can't modify non-existent alias testModifyMetadataCAR
	at __randomizedtesting.SeedInfo.seed([E1F4D5F761FF3F81:FDA694BCDC249FD9]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
	at org.apache.solr.cloud.AliasIntegrationTest.testModifyMetadataCAR(AliasIntegrationTest.java:261)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 13825 lines...]
   [junit4] Suite: org.apache.solr.cloud.AliasIntegrationTest
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.AliasIntegrationTest_E1F4D5F761FF3F81-001/init-core-data-001
   [junit4]   2> 2011921 WARN  (SUITE-AliasIntegrationTest-seed#[E1F4D5F761FF3F81]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
   [junit4]   2> 2011921 INFO  (SUITE-AliasIntegrationTest-seed#[E1F4D5F761FF3F81]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 2011923 INFO  (SUITE-AliasIntegrationTest-seed#[E1F4D5F761FF3F81]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 2011923 INFO  (SUITE-AliasIntegrationTest-seed#[E1F4D5F761FF3F81]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 2011924 INFO  (SUITE-AliasIntegrationTest-seed#[E1F4D5F761FF3F81]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.AliasIntegrationTest_E1F4D5F761FF3F81-001/tempDir-001
   [junit4]   2> 2011924 INFO  (SUITE-AliasIntegrationTest-seed#[E1F4D5F761FF3F81]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2011924 INFO  (Thread-3932) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2011924 INFO  (Thread-3932) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2011926 ERROR (Thread-3932) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 2012028 INFO  (SUITE-AliasIntegrationTest-seed#[E1F4D5F761FF3F81]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:37903
   [junit4]   2> 2012031 INFO  (zkConnectionManagerCallback-5538-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2012036 INFO  (jetty-launcher-5535-thread-1) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T00:27:37+03:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 2012036 INFO  (jetty-launcher-5535-thread-2) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T00:27:37+03:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 2012037 INFO  (jetty-launcher-5535-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 2012037 INFO  (jetty-launcher-5535-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 2012037 INFO  (jetty-launcher-5535-thread-1) [    ] o.e.j.s.session Scavenging every 600000ms
   [junit4]   2> 2012037 INFO  (jetty-launcher-5535-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@67479904{/solr,null,AVAILABLE}
   [junit4]   2> 2012038 INFO  (jetty-launcher-5535-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 2012038 INFO  (jetty-launcher-5535-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 2012038 INFO  (jetty-launcher-5535-thread-2) [    ] o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 2012038 INFO  (jetty-launcher-5535-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1eb6fecc{/solr,null,AVAILABLE}
   [junit4]   2> 2012039 INFO  (jetty-launcher-5535-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@4121ee6a{HTTP/1.1,[http/1.1]}{127.0.0.1:48928}
   [junit4]   2> 2012039 INFO  (jetty-launcher-5535-thread-1) [    ] o.e.j.s.Server Started @2014644ms
   [junit4]   2> 2012039 INFO  (jetty-launcher-5535-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@4a2b5fd4{HTTP/1.1,[http/1.1]}{127.0.0.1:43196}
   [junit4]   2> 2012039 INFO  (jetty-launcher-5535-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=48928}
   [junit4]   2> 2012039 INFO  (jetty-launcher-5535-thread-2) [    ] o.e.j.s.Server Started @2014644ms
   [junit4]   2> 2012039 INFO  (jetty-launcher-5535-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=43196}
   [junit4]   2> 2012039 ERROR (jetty-launcher-5535-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2012039 ERROR (jetty-launcher-5535-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2012040 INFO  (jetty-launcher-5535-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 2012040 INFO  (jetty-launcher-5535-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 2012040 INFO  (jetty-launcher-5535-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2012040 INFO  (jetty-launcher-5535-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2012040 INFO  (jetty-launcher-5535-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2012040 INFO  (jetty-launcher-5535-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2012040 INFO  (jetty-launcher-5535-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-03-02T15:45:55.842Z
   [junit4]   2> 2012040 INFO  (jetty-launcher-5535-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-03-02T15:45:55.842Z
   [junit4]   2> 2012042 INFO  (zkConnectionManagerCallback-5540-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2012043 INFO  (zkConnectionManagerCallback-5542-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2012043 INFO  (jetty-launcher-5535-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2012043 INFO  (jetty-launcher-5535-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2012050 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1000881a79d0002, likely client has closed socket
   [junit4]   2> 2012050 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1000881a79d0001, likely client has closed socket
   [junit4]   2> 2012053 INFO  (jetty-launcher-5535-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37903/solr
   [junit4]   2> 2012054 INFO  (zkConnectionManagerCallback-5546-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2012058 INFO  (jetty-launcher-5535-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37903/solr
   [junit4]   2> 2012059 INFO  (zkConnectionManagerCallback-5548-thread-1-processing-n:127.0.0.1:48928_solr) [n:127.0.0.1:48928_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2012061 INFO  (zkConnectionManagerCallback-5552-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2012070 INFO  (zkConnectionManagerCallback-5554-thread-1-processing-n:127.0.0.1:43196_solr) [n:127.0.0.1:43196_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2012105 INFO  (jetty-launcher-5535-thread-2) [n:127.0.0.1:43196_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2012106 INFO  (jetty-launcher-5535-thread-2) [n:127.0.0.1:43196_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:43196_solr
   [junit4]   2> 2012108 INFO  (jetty-launcher-5535-thread-2) [n:127.0.0.1:43196_solr    ] o.a.s.c.Overseer Overseer (id=72066946993815558-127.0.0.1:43196_solr-n_0000000000) starting
   [junit4]   2> 2012125 INFO  (jetty-launcher-5535-thread-2) [n:127.0.0.1:43196_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43196_solr
   [junit4]   2> 2012128 INFO  (OverseerStateUpdate-72066946993815558-127.0.0.1:43196_solr-n_0000000000) [n:127.0.0.1:43196_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2012191 INFO  (jetty-launcher-5535-thread-1) [n:127.0.0.1:48928_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2012193 INFO  (jetty-launcher-5535-thread-1) [n:127.0.0.1:48928_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2012195 INFO  (jetty-launcher-5535-thread-1) [n:127.0.0.1:48928_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48928_solr
   [junit4]   2> 2012199 INFO  (zkCallback-5547-thread-1-processing-n:127.0.0.1:48928_solr) [n:127.0.0.1:48928_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2012224 INFO  (zkCallback-5553-thread-2-processing-n:127.0.0.1:43196_solr) [n:127.0.0.1:43196_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2012282 INFO  (jetty-launcher-5535-thread-2) [n:127.0.0.1:43196_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43196.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2f1ec2fa
   [junit4]   2> 2012290 INFO  (jetty-launcher-5535-thread-2) [n:127.0.0.1:43196_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43196.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2f1ec2fa
   [junit4]   2> 2012291 INFO  (jetty-launcher-5535-thread-2) [n:127.0.0.1:43196_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43196.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2f1ec2fa
   [junit4]   2> 2012292 INFO  (jetty-launcher-5535-thread-2) [n:127.0.0.1:43196_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.AliasIntegrationTest_E1F4D5F761FF3F81-001/tempDir-001/node2/.
   [junit4]   2> 2012311 INFO  (jetty-launcher-5535-thread-1) [n:127.0.0.1:48928_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48928.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2f1ec2fa
   [junit4]   2> 2012318 INFO  (jetty-launcher-5535-thread-1) [n:127.0.0.1:48928_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48928.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2f1ec2fa
   [junit4]   2> 2012319 INFO  (jetty-launcher-5535-thread-1) [n:127.0.0.1:48928_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48928.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2f1ec2fa
   [junit4]   2> 2012319 INFO  (jetty-launcher-5535-thread-1) [n:127.0.0.1:48928_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.AliasIntegrationTest_E1F4D5F761FF3F81-001/tempDir-001/node1/.
   [junit4]   2> 2012343 INFO  (zkConnectionManagerCallback-5560-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2012347 INFO  (zkConnectionManagerCallback-5564-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2012348 INFO  (SUITE-AliasIntegrationTest-seed#[E1F4D5F761FF3F81]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2012353 INFO  (SUITE-AliasIntegrationTest-seed#[E1F4D5F761FF3F81]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37903/solr ready
   [junit4]   2> 2012408 INFO  (TEST-AliasIntegrationTest.testMetadata-seed#[E1F4D5F761FF3F81]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testMetadata
   [junit4]   2> 2012413 INFO  (qtp92139696-18416) [n:127.0.0.1:43196_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :clusterstatus with params action=CLUSTERSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2012414 INFO  (qtp92139696-18416) [n:127.0.0.1:43196_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=CLUSTERSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2012418 INFO  (qtp92139696-18430) [n:127.0.0.1:43196_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf&name=collection1meta&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2012421 INFO  (OverseerThreadFactory-7226-thread-1-processing-n:127.0.0.1:43196_solr) [n:127.0.0.1:43196_solr    ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1meta
   [junit4]   2> 2012550 INFO  (OverseerStateUpdate-72066946993815558-127.0.0.1:43196_solr-n_0000000000) [n:127.0.0.1:43196_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"collection1meta",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"collection1meta_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:48928/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 2012554 INFO  (OverseerStateUpdate-72066946993815558-127.0.0.1:43196_solr-n_0000000000) [n:127.0.0.1:43196_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"collection1meta",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"collection1meta_shard2_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:43196/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 2012766 INFO  (qtp359127126-18415) [n:127.0.0.1:48928_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=collection1meta_shard1_replica_n1&action=CREATE&numShards=2&collection=collection1meta&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 2012766 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=collection1meta_shard2_replica_n2&action=CREATE&numShards=2&collection=collection1meta&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 2012766 INFO  (qtp359127126-18415) [n:127.0.0.1:48928_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2012768 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2012884 INFO  (zkCallback-5547-thread-1-processing-n:127.0.0.1:48928_solr) [n:127.0.0.1:48928_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1meta/state.json] for collection [collection1meta] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2012884 INFO  (zkCallback-5547-thread-2-processing-n:127.0.0.1:48928_solr) [n:127.0.0.1:48928_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1meta/state.json] for collection [collection1meta] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2012884 INFO  (zkCallback-5553-thread-1-processing-n:127.0.0.1:43196_solr) [n:127.0.0.1:43196_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1meta/state.json] for collection [collection1meta] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2012884 INFO  (zkCallback-5553-thread-2-processing-n:127.0.0.1:43196_solr) [n:127.0.0.1:43196_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1meta/state.json] for collection [collection1meta] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2013798 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 2013798 INFO  (qtp359127126-18415) [n:127.0.0.1:48928_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 2013806 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.s.IndexSchema [collection1meta_shard2_replica_n2] Schema name=minimal
   [junit4]   2> 2013806 INFO  (qtp359127126-18415) [n:127.0.0.1:48928_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.s.IndexSchema [collection1meta_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 2013809 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 2013809 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'collection1meta_shard2_replica_n2' using configuration from collection collection1meta, trusted=true
   [junit4]   2> 2013809 INFO  (qtp359127126-18415) [n:127.0.0.1:48928_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 2013809 INFO  (qtp359127126-18415) [n:127.0.0.1:48928_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'collection1meta_shard1_replica_n1' using configuration from collection collection1meta, trusted=true
   [junit4]   2> 2013809 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43196.solr.core.collection1meta.shard2.replica_n2' (registry 'solr.core.collection1meta.shard2.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2f1ec2fa
   [junit4]   2> 2013809 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2013809 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.c.SolrCore [[collection1meta_shard2_replica_n2] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.AliasIntegrationTest_E1F4D5F761FF3F81-001/tempDir-001/node2/collection1meta_shard2_replica_n2], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.AliasIntegrationTest_E1F4D5F761FF3F81-001/tempDir-001/node2/./collection1meta_shard2_replica_n2/data/]
   [junit4]   2> 2013809 INFO  (qtp359127126-18415) [n:127.0.0.1:48928_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48928.solr.core.collection1meta.shard1.replica_n1' (registry 'solr.core.collection1meta.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2f1ec2fa
   [junit4]   2> 2013810 INFO  (qtp359127126-18415) [n:127.0.0.1:48928_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2013810 INFO  (qtp359127126-18415) [n:127.0.0.1:48928_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.c.SolrCore [[collection1meta_shard1_replica_n1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.AliasIntegrationTest_E1F4D5F761FF3F81-001/tempDir-001/node1/collection1meta_shard1_replica_n1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.AliasIntegrationTest_E1F4D5F761FF3F81-001/tempDir-001/node1/./collection1meta_shard1_replica_n1/data/]
   [junit4]   2> 2013901 INFO  (qtp359127126-18415) [n:127.0.0.1:48928_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2013901 INFO  (qtp359127126-18415) [n:127.0.0.1:48928_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2013902 INFO  (qtp359127126-18415) [n:127.0.0.1:48928_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2013902 INFO  (qtp359127126-18415) [n:127.0.0.1:48928_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2013904 INFO  (qtp359127126-18415) [n:127.0.0.1:48928_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@50b9217a[collection1meta_shard1_replica_n1] main]
   [junit4]   2> 2013906 INFO  (qtp359127126-18415) [n:127.0.0.1:48928_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 2013907 INFO  (qtp359127126-18415) [n:127.0.0.1:48928_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 2013907 INFO  (qtp359127126-18415) [n:127.0.0.1:48928_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 2013908 INFO  (qtp359127126-18415) [n:127.0.0.1:48928_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1593841347681320960
   [junit4]   2> 2013908 INFO  (searcherExecutor-7232-thread-1-processing-n:127.0.0.1:48928_solr x:collection1meta_shard1_replica_n1 s:shard1 c:collection1meta r:core_node3) [n:127.0.0.1:48928_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.c.SolrCore [collection1meta_shard1_replica_n1] Registered new searcher Searcher@50b9217a[collection1meta_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2013911 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2013911 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2013912 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2013912 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2013916 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@732d6998[collection1meta_shard2_replica_n2] main]
   [junit4]   2> 2013918 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 2013918 INFO  (qtp359127126-18415) [n:127.0.0.1:48928_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2013918 INFO  (qtp359127126-18415) [n:127.0.0.1:48928_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2013918 INFO  (qtp359127126-18415) [n:127.0.0.1:48928_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:48928/solr/collection1meta_shard1_replica_n1/
   [junit4]   2> 2013918 INFO  (qtp359127126-18415) [n:127.0.0.1:48928_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2013919 INFO  (qtp359127126-18415) [n:127.0.0.1:48928_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:48928/solr/collection1meta_shard1_replica_n1/ has no replicas
   [junit4]   2> 2013919 INFO  (qtp359127126-18415) [n:127.0.0.1:48928_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 2013919 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 2013931 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 2013931 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1593841347705438208
   [junit4]   2> 2013932 INFO  (searcherExecutor-7231-thread-1-processing-n:127.0.0.1:43196_solr x:collection1meta_shard2_replica_n2 s:shard2 c:collection1meta r:core_node4) [n:127.0.0.1:43196_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.c.SolrCore [collection1meta_shard2_replica_n2] Registered new searcher Searcher@732d6998[collection1meta_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2013936 INFO  (qtp359127126-18415) [n:127.0.0.1:48928_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:48928/solr/collection1meta_shard1_replica_n1/ shard1
   [junit4]   2> 2013945 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2013945 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2013945 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:43196/solr/collection1meta_shard2_replica_n2/
   [junit4]   2> 2013945 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2013945 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.c.SyncStrategy http://127.0.0.1:43196/solr/collection1meta_shard2_replica_n2/ has no replicas
   [junit4]   2> 2013945 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 2013951 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:43196/solr/collection1meta_shard2_replica_n2/ shard2
   [junit4]   2> 2014053 INFO  (zkCallback-5547-thread-2-processing-n:127.0.0.1:48928_solr) [n:127.0.0.1:48928_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1meta/state.json] for collection [collection1meta] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2014054 INFO  (zkCallback-5553-thread-1-processing-n:127.0.0.1:43196_solr) [n:127.0.0.1:43196_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1meta/state.json] for collection [collection1meta] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2014054 INFO  (zkCallback-5553-thread-2-processing-n:127.0.0.1:43196_solr) [n:127.0.0.1:43196_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1meta/state.json] for collection [collection1meta] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2014053 INFO  (zkCallback-5547-thread-1-processing-n:127.0.0.1:48928_solr) [n:127.0.0.1:48928_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1meta/state.json] for collection [collection1meta] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2014093 INFO  (qtp359127126-18415) [n:127.0.0.1:48928_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2014096 INFO  (qtp359127126-18415) [n:127.0.0.1:48928_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=collection1meta_shard1_replica_n1&action=CREATE&numShards=2&collection=collection1meta&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1330
   [junit4]   2> 2014103 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2014105 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=collection1meta_shard2_replica_n2&action=CREATE&numShards=2&collection=collection1meta&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1339
   [junit4]   2> 2014108 INFO  (qtp92139696-18430) [n:127.0.0.1:43196_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 2014215 INFO  (zkCallback-5553-thread-2-processing-n:127.0.0.1:43196_solr) [n:127.0.0.1:43196_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1meta/state.json] for collection [collection1meta] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2014216 INFO  (zkCallback-5553-thread-1-processing-n:127.0.0.1:43196_solr) [n:127.0.0.1:43196_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1meta/state.json] for collection [collection1meta] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2014216 INFO  (zkCallback-5547-thread-1-processing-n:127.0.0.1:48928_solr) [n:127.0.0.1:48928_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1meta/state.json] for collection [collection1meta] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2014216 INFO  (zkCallback-5547-thread-2-processing-n:127.0.0.1:48928_solr) [n:127.0.0.1:48928_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1meta/state.json] for collection [collection1meta] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2014423 INFO  (OverseerCollectionConfigSetProcessor-72066946993815558-127.0.0.1:43196_solr-n_0000000000) [n:127.0.0.1:43196_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 2015113 INFO  (qtp92139696-18430) [n:127.0.0.1:43196_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf&name=collection1meta&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=2696
   [junit4]   2> 2015114 INFO  (qtp92139696-18426) [n:127.0.0.1:43196_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf&name=collection2meta&nrtReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2015116 INFO  (OverseerThreadFactory-7226-thread-2-processing-n:127.0.0.1:43196_solr) [n:127.0.0.1:43196_solr    ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection2meta
   [junit4]   2> 2015336 INFO  (OverseerStateUpdate-72066946993815558-127.0.0.1:43196_solr-n_0000000000) [n:127.0.0.1:43196_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"collection2meta",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"collection2meta_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:43196/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 2015538 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf&newCollection=true&name=collection2meta_shard1_replica_n1&action=CREATE&numShards=1&collection=collection2meta&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 2015643 INFO  (zkCallback-5553-thread-1-processing-n:127.0.0.1:43196_solr) [n:127.0.0.1:43196_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection2meta/state.json] for collection [collection2meta] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2015643 INFO  (zkCallback-5553-thread-2-processing-n:127.0.0.1:43196_solr) [n:127.0.0.1:43196_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection2meta/state.json] for collection [collection2meta] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2016551 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 2016555 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.s.IndexSchema [collection2meta_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 2016557 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 2016557 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'collection2meta_shard1_replica_n1' using configuration from collection collection2meta, trusted=true
   [junit4]   2> 2016557 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43196.solr.core.collection2meta.shard1.replica_n1' (registry 'solr.core.collection2meta.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2f1ec2fa
   [junit4]   2> 2016557 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2016557 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.c.SolrCore [[collection2meta_shard1_replica_n1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.AliasIntegrationTest_E1F4D5F761FF3F81-001/tempDir-001/node2/collection2meta_shard1_replica_n1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.AliasIntegrationTest_E1F4D5F761FF3F81-001/tempDir-001/node2/./collection2meta_shard1_replica_n1/data/]
   [junit4]   2> 2016600 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2016600 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2016601 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2016601 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2016602 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@495bda06[collection2meta_shard1_replica_n1] main]
   [junit4]   2> 2016603 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 2016603 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 2016604 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 2016604 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1593841350508281856
   [junit4]   2> 2016605 INFO  (searcherExecutor-7241-thread-1-processing-n:127.0.0.1:43196_solr x:collection2meta_shard1_replica_n1 s:shard1 c:collection2meta r:core_node2) [n:127.0.0.1:43196_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.c.SolrCore [collection2meta_shard1_replica_n1] Registered new searcher Searcher@495bda06[collection2meta_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2016616 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2016616 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2016616 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:43196/solr/collection2meta_shard1_replica_n1/
   [junit4]   2> 2016616 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2016616 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:43196/solr/collection2meta_shard1_replica_n1/ has no replicas
   [junit4]   2> 2016616 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 2016621 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:43196/solr/collection2meta_shard1_replica_n1/ shard1
   [junit4]   2> 2016723 INFO  (zkCallback-5553-thread-3-processing-n:127.0.0.1:43196_solr) [n:127.0.0.1:43196_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection2meta/state.json] for collection [collection2meta] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2016723 INFO  (zkCallback-5553-thread-2-processing-n:127.0.0.1:43196_solr) [n:127.0.0.1:43196_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection2meta/state.json] for collection [collection2meta] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2016773 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2016775 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf&newCollection=true&name=collection2meta_shard1_replica_n1&action=CREATE&numShards=1&collection=collection2meta&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1237
   [junit4]   2> 2016781 INFO  (qtp92139696-18426) [n:127.0.0.1:43196_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 2016877 INFO  (zkCallback-5553-thread-2-processing-n:127.0.0.1:43196_solr) [n:127.0.0.1:43196_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection2meta/state.json] for collection [collection2meta] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2016877 INFO  (zkCallback-5553-thread-3-processing-n:127.0.0.1:43196_solr) [n:127.0.0.1:43196_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection2meta/state.json] for collection [collection2meta] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2017124 INFO  (OverseerCollectionConfigSetProcessor-72066946993815558-127.0.0.1:43196_solr-n_0000000000) [n:127.0.0.1:43196_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 2017782 INFO  (qtp92139696-18426) [n:127.0.0.1:43196_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf&name=collection2meta&nrtReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=2668
   [junit4]   2> 2017798 INFO  (zkConnectionManagerCallback-5567-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2017804 INFO  (TEST-AliasIntegrationTest.testMetadata-seed#[E1F4D5F761FF3F81]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2017806 INFO  (TEST-AliasIntegrationTest.testMetadata-seed#[E1F4D5F761FF3F81]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testMetadata
   [junit4]   2> 2017809 INFO  (qtp92139696-18429) [n:127.0.0.1:43196_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :listaliases with params action=LISTALIASES&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2017809 INFO  (qtp92139696-18429) [n:127.0.0.1:43196_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LISTALIASES&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2017811 INFO  (qtp92139696-18426) [n:127.0.0.1:43196_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :deletealias with params name=meta1&action=DELETEALIAS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2017818 INFO  (qtp92139696-18426) [n:127.0.0.1:43196_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=meta1&action=DELETEALIAS&wt=javabin&version=2} status=0 QTime=6
   [junit4]   2> 2017819 INFO  (qtp92139696-18429) [n:127.0.0.1:43196_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :deletealias with params name=meta3&action=DELETEALIAS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2017822 INFO  (OverseerCollectionConfigSetProcessor-72066946993815558-127.0.0.1:43196_solr-n_0000000000) [n:127.0.0.1:43196_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 2017826 INFO  (qtp92139696-18429) [n:127.0.0.1:43196_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=meta3&action=DELETEALIAS&wt=javabin&version=2} status=0 QTime=6
   [junit4]   2> 2017827 INFO  (TEST-AliasIntegrationTest.testMetadata-seed#[E1F4D5F761FF3F81]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2017829 INFO  (qtp92139696-18426) [n:127.0.0.1:43196_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params name=collection2meta&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2017832 INFO  (OverseerThreadFactory-7226-thread-5-processing-n:127.0.0.1:43196_solr) [n:127.0.0.1:43196_solr    ] o.a.s.c.a.c.OverseerCollectionMessageHandler Executing Collection Cmd=action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true, asyncId=null
   [junit4]   2> 2017833 INFO  (OverseerCollectionConfigSetProcessor-72066946993815558-127.0.0.1:43196_solr-n_0000000000) [n:127.0.0.1:43196_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 2017833 INFO  (qtp92139696-18429) [n:127.0.0.1:43196_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection2meta.shard1.replica_n1, tag=null
   [junit4]   2> 2017833 INFO  (qtp92139696-18429) [n:127.0.0.1:43196_solr    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@b3b347e: rootName = solr_43196, domain = solr.core.collection2meta.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.collection2meta.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@7374116f
   [junit4]   2> 2017845 INFO  (qtp92139696-18429) [n:127.0.0.1:43196_solr    ] o.a.s.c.SolrCore [collection2meta_shard1_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@62169e13
   [junit4]   2> 2017845 INFO  (qtp92139696-18429) [n:127.0.0.1:43196_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection2meta.shard1.replica_n1, tag=1645649427
   [junit4]   2> 2017845 INFO  (qtp92139696-18429) [n:127.0.0.1:43196_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection2meta.shard1.leader, tag=1645649427
   [junit4]   2> 2017848 INFO  (qtp92139696-18429) [n:127.0.0.1:43196_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=collection2meta_shard1_replica_n1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=15
   [junit4]   2> 2018569 INFO  (qtp92139696-18426) [n:127.0.0.1:43196_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=collection2meta&action=DELETE&wt=javabin&version=2} status=0 QTime=739
   [junit4]   2> 2018569 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params name=collection1meta&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2018571 INFO  (OverseerThreadFactory-7226-thread-5-processing-n:127.0.0.1:43196_solr) [n:127.0.0.1:43196_solr    ] o.a.s.c.a.c.OverseerCollectionMessageHandler Executing Collection Cmd=action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true, asyncId=null
   [junit4]   2> 2018572 INFO  (OverseerCollectionConfigSetProcessor-72066946993815558-127.0.0.1:43196_solr-n_0000000000) [n:127.0.0.1:43196_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000008 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 2018572 INFO  (qtp92139696-18426) [n:127.0.0.1:43196_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1meta.shard2.replica_n2, tag=null
   [junit4]   2> 2018572 INFO  (qtp92139696-18426) [n:127.0.0.1:43196_solr    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@63fb512b: rootName = solr_43196, domain = solr.core.collection1meta.shard2.replica_n2, service url = null, agent id = null] for registry solr.core.collection1meta.shard2.replica_n2 / com.codahale.metrics.MetricRegistry@5cd06a98
   [junit4]   2> 2018573 INFO  (qtp359127126-18418) [n:127.0.0.1:48928_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1meta.shard1.replica_n1, tag=null
   [junit4]   2> 2018574 INFO  (qtp359127126-18418) [n:127.0.0.1:48928_solr    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@e7aff06: rootName = solr_48928, domain = solr.core.collection1meta.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.collection1meta.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@649909
   [junit4]   2> 2018598 INFO  (qtp92139696-18426) [n:127.0.0.1:43196_solr    ] o.a.s.c.SolrCore [collection1meta_shard2_replica_n2]  CLOSING SolrCore org.apache.solr.core.SolrCore@7c471976
   [junit4]   2> 2018598 INFO  (qtp92139696-18426) [n:127.0.0.1:43196_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1meta.shard2.replica_n2, tag=2085034358
   [junit4]   2> 2018598 INFO  (qtp92139696-18426) [n:127.0.0.1:43196_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1meta.shard2.leader, tag=2085034358
   [junit4]   2> 2018602 INFO  (qtp92139696-18426) [n:127.0.0.1:43196_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=collection1meta_shard2_replica_n2&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=30
   [junit4]   2> 2018603 INFO  (qtp359127126-18418) [n:127.0.0.1:48928_solr    ] o.a.s.c.SolrCore [collection1meta_shard1_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@486be90a
   [junit4]   2> 2018603 INFO  (qtp359127126-18418) [n:127.0.0.1:48928_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1meta.shard1.replica_n1, tag=1215031562
   [junit4]   2> 2018603 INFO  (qtp359127126-18418) [n:127.0.0.1:48928_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1meta.shard1.leader, tag=1215031562
   [junit4]   2> 2018606 INFO  (qtp359127126-18418) [n:127.0.0.1:48928_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=collection1meta_shard1_replica_n1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=32
   [junit4]   2> 2019321 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=collection1meta&action=DELETE&wt=javabin&version=2} status=0 QTime=752
   [junit4]   2> 2019365 INFO  (TEST-AliasIntegrationTest.testDeleteOneOfTwoCollectionsAliased-seed#[E1F4D5F761FF3F81]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testDeleteOneOfTwoCollectionsAliased
   [junit4]   2> 2019368 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :clusterstatus with params action=CLUSTERSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2019369 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=CLUSTERSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2019378 INFO  (qtp92139696-18429) [n:127.0.0.1:43196_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf&name=collection_one&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2019386 INFO  (OverseerThreadFactory-7226-thread-5-processing-n:127.0.0.1:43196_solr) [n:127.0.0.1:43196_solr    ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection_one
   [junit4]   2> 2019396 INFO  (OverseerCollectionConfigSetProcessor-72066946993815558-127.0.0.1:43196_solr-n_0000000000) [n:127.0.0.1:43196_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000010 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 2019634 INFO  (OverseerStateUpdate-72066946993815558-127.0.0.1:43196_solr-n_0000000000) [n:127.0.0.1:43196_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"collection_one",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"collection_one_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:48928/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 2019637 INFO  (OverseerStateUpdate-72066946993815558-127.0.0.1:43196_solr-n_0000000000) [n:127.0.0.1:43196_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"collection_one",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"collection_one_shard2_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:43196/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 2019839 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=collection_one_shard2_replica_n2&action=CREATE&numShards=2&collection=collection_one&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 2019841 INFO  (qtp359127126-18420) [n:127.0.0.1:48928_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=collection_one_shard1_replica_n1&action=CREATE&numShards=2&collection=collection_one&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 2019953 INFO  (zkCallback-5553-thread-2-processing-n:127.0.0.1:43196_solr) [n:127.0.0.1:43196_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_one/state.json] for collection [collection_one] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2019953 INFO  (zkCallback-5553-thread-3-processing-n:127.0.0.1:43196_solr) [n:127.0.0.1:43196_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_one/state.json] for collection [collection_one] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2019953 INFO  (zkCallback-5547-thread-1-processing-n:127.0.0.1:48928_solr) [n:127.0.0.1:48928_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_one/state.json] for collection [collection_one] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2019953 INFO  (zkCallback-5547-thread-2-processing-n:127.0.0.1:48928_solr) [n:127.0.0.1:48928_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_one/state.json] for collection [collection_one] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2020865 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 2020880 INFO  (qtp359127126-18420) [n:127.0.0.1:48928_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 2020888 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.s.IndexSchema [collection_one_shard2_replica_n2] Schema name=minimal
   [junit4]   2> 2020889 INFO  (qtp359127126-18420) [n:127.0.0.1:48928_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.s.IndexSchema [collection_one_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 2020890 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 2020890 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'collection_one_shard2_replica_n2' using configuration from collection collection_one, trusted=true
   [junit4]   2> 2020891 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43196.solr.core.collection_one.shard2.replica_n2' (registry 'solr.core.collection_one.shard2.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2f1ec2fa
   [junit4]   2> 2020891 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2020891 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.c.SolrCore [[collection_one_shard2_replica_n2] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.AliasIntegrationTest_E1F4D5F761FF3F81-001/tempDir-001/node2/collection_one_shard2_replica_n2], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.AliasIntegrationTest_E1F4D5F761FF3F81-001/tempDir-001/node2/./collection_one_shard2_replica_n2/data/]
   [junit4]   2> 2020891 INFO  (qtp359127126-18420) [n:127.0.0.1:48928_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 2020892 INFO  (qtp359127126-18420) [n:127.0.0.1:48928_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'collection_one_shard1_replica_n1' using configuration from collection collection_one, trusted=true
   [junit4]   2> 2020892 INFO  (qtp359127126-18420) [n:127.0.0.1:48928_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48928.solr.core.collection_one.shard1.replica_n1' (registry 'solr.core.collection_one.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2f1ec2fa
   [junit4]   2> 2020892 INFO  (qtp359127126-18420) [n:127.0.0.1:48928_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2020892 INFO  (qtp359127126-18420) [n:127.0.0.1:48928_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.c.SolrCore [[collection_one_shard1_replica_n1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.AliasIntegrationTest_E1F4D5F761FF3F81-001/tempDir-001/node1/collection_one_shard1_replica_n1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.AliasIntegrationTest_E1F4D5F761FF3F81-001/tempDir-001/node1/./collection_one_shard1_replica_n1/data/]
   [junit4]   2> 2020945 INFO  (qtp359127126-18420) [n:127.0.0.1:48928_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2020945 INFO  (qtp359127126-18420) [n:127.0.0.1:48928_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2020946 INFO  (qtp359127126-18420) [n:127.0.0.1:48928_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2020946 INFO  (qtp359127126-18420) [n:127.0.0.1:48928_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2020948 INFO  (qtp359127126-18420) [n:127.0.0.1:48928_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2192b709[collection_one_shard1_replica_n1] main]
   [junit4]   2> 2020949 INFO  (qtp359127126-18420) [n:127.0.0.1:48928_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 2020949 INFO  (qtp359127126-18420) [n:127.0.0.1:48928_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 2020950 INFO  (qtp359127126-18420) [n:127.0.0.1:48928_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 2020950 INFO  (qtp359127126-18420) [n:127.0.0.1:48928_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1593841355065393152
   [junit4]   2> 2020951 INFO  (searcherExecutor-7247-thread-1-processing-n:127.0.0.1:48928_solr x:collection_one_shard1_replica_n1 s:shard1 c:collection_one r:core_node3) [n:127.0.0.1:48928_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.c.SolrCore [collection_one_shard1_replica_n1] Registered new searcher Searcher@2192b709[collection_one_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2020955 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2020955 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2020956 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2020956 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2020958 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@eb1d7cd[collection_one_shard2_replica_n2] main]
   [junit4]   2> 2020960 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 2020960 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 2020962 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 2020962 INFO  (qtp359127126-18420) [n:127.0.0.1:48928_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2020962 INFO  (qtp359127126-18420) [n:127.0.0.1:48928_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2020963 INFO  (qtp359127126-18420) [n:127.0.0.1:48928_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:48928/solr/collection_one_shard1_replica_n1/
   [junit4]   2> 2020963 INFO  (qtp359127126-18420) [n:127.0.0.1:48928_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2020963 INFO  (qtp359127126-18420) [n:127.0.0.1:48928_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:48928/solr/collection_one_shard1_replica_n1/ has no replicas
   [junit4]   2> 2020963 INFO  (qtp359127126-18420) [n:127.0.0.1:48928_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 2020963 INFO  (searcherExecutor-7246-thread-1-processing-n:127.0.0.1:43196_solr x:collection_one_shard2_replica_n2 s:shard2 c:collection_one r:core_node4) [n:127.0.0.1:43196_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.c.SolrCore [collection_one_shard2_replica_n2] Registered new searcher Searcher@eb1d7cd[collection_one_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2020963 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1593841355079024640
   [junit4]   2> 2020968 INFO  (qtp359127126-18420) [n:127.0.0.1:48928_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:48928/solr/collection_one_shard1_replica_n1/ shard1
   [junit4]   2> 2020973 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2020973 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2020973 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:43196/solr/collection_one_shard2_replica_n2/
   [junit4]   2> 2020973 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2020973 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.c.SyncStrategy http://127.0.0.1:43196/solr/collection_one_shard2_replica_n2/ has no replicas
   [junit4]   2> 2020973 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 2020982 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:43196/solr/collection_one_shard2_replica_n2/ shard2
   [junit4]   2> 2021084 INFO  (zkCallback-5553-thread-3-processing-n:127.0.0.1:43196_solr) [n:127.0.0.1:43196_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_one/state.json] for collection [collection_one] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2021085 INFO  (zkCallback-5547-thread-1-processing-n:127.0.0.1:48928_solr) [n:127.0.0.1:48928_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_one/state.json] for collection [collection_one] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2021085 INFO  (zkCallback-5553-thread-2-processing-n:127.0.0.1:43196_solr) [n:127.0.0.1:43196_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_one/state.json] for collection [collection_one] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2021084 INFO  (zkCallback-5547-thread-2-processing-n:127.0.0.1:48928_solr) [n:127.0.0.1:48928_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_one/state.json] for collection [collection_one] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2021125 INFO  (qtp359127126-18420) [n:127.0.0.1:48928_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2021128 INFO  (qtp359127126-18420) [n:127.0.0.1:48928_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=collection_one_shard1_replica_n1&action=CREATE&numShards=2&collection=collection_one&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1286
   [junit4]   2> 2021134 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2021136 INFO  (qtp92139696-18425) [n:127.0.0.1:43196_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=collection_one_shard2_replica_n2&action=CREATE&numShards=2&collection=collection_one&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1297
   [junit4]   2> 2021138 INFO  (qtp92139696-18429) [n:127.0.0.1:43196_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 2021238 INFO  (zkCallback-5547-thread-2-processing-n:127.0.0.1:48928_solr) [n:127.0.0.1:48928_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_one/state.json] for collection [collection_one] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2021238 INFO  (zkCallback-5553-thread-3-processing-n:127.0.0.1:43196_solr) [n:127.0.0.1:43196_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_one/state.json] for collection [collection_one] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2021238 INFO  (zkCallback-5553-thread-2-processing-n:127.0.0.1:43196_solr) [n:127.0.0.1:43196_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_one/state.json] for collection [collection_one] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2021238 INFO  (zkCallback-5547-thread-1-processing-n:127.0.0.1:48928_solr) [n:127.0.0.1:48928_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_one/state.json] for collection [collection_one] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2021398 INFO  (OverseerCollectionConfigSetProcessor-72066946993815558-127.0.0.1:43196_solr-n_0000000000) [n:127.0.0.1:43196_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000012 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 2022139 INFO  (qtp92139696-18429) [n:127.0.0.1:43196_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf&name=collection_one&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=2761
   [junit4]   2> 2022145 INFO  (qtp92139696-18426) [n:127.0.0.1:43196_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf&name=collection_two&nrtReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2022148 INFO  (OverseerThreadFactory-7226-thread-5-processing-n:127.0.0.1:43196_solr) [n:127.0.0.1:43196_solr    ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection_two
   [junit4]   2> 2022362 INFO  (OverseerStateUpdate-72066946993815558-127.0.0.1:43196_solr-n_0000000000) [n:127.0.0.1:43196_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"collection_two",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"collection_two_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:48928/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 2022563 INFO  (qtp359127126-18418) [n:127.0.0.1:48928_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf&newCollection=true&name=collection_two_shard1_replica_n1&action=CREATE&numShards=1&collection=collection_two&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 2022668 INFO  (zkCallback-5547-thread-1-processing-n:127.0.0.1:48928_solr) [n:127.0.0.1:48928_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_two/state.json] for collection [collection_two] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2022668 INFO  (zkCallback-5547-thread-2-processing-n:127.0.0.1:48928_solr) [n:127.0.0.1:48928_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_two/state.json] for collection [collection_two] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2023590 INFO  (qtp359127126-18418) [n:127.0.0.1:48928_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 2023596 INFO  (qtp359127126-18418) [n:127.0.0.1:48928_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.s.IndexSchema [collection_two_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 2023597 INFO  (qtp359127126-18418) [n:127.0.0.1:48928_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 2023597 INFO  (qtp359127126-18418) [n:127.0.0.1:48928_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'collection_two_shard1_replica_n1' using configuration from collection collection_two, trusted=true
   [junit4]   2> 2023598 INFO  (qtp359127126-18418) [n:127.0.0.1:48928_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48928.solr.core.collection_two.shard1.replica_n1' (registry 'solr.core.collection_two.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2f1ec2fa
   [junit4]   2> 2023598 INFO  (qtp359127126-18418) [n:127.0.0.1:48928_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 2023598 INFO  (qtp359127126-18418) [n:127.0.0.1:48928_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.c.SolrCore [[collection_two_shard1_replica_n1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.AliasIntegrationTest_E1F4D5F761FF3F81-001/tempDir-001/node1/collection_two_shard1_replica_n1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.AliasIntegrationTest_E1F4D5F761FF3F81-001/tempDir-001/node1/./collection_two_shard1_replica_n1/data/]
   [junit4]   2> 2023640 INFO  (qtp359127126-18418) [n:127.0.0.1:48928_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2023640 INFO  (qtp359127126-18418) [n:127.0.0.1:48928_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2023641 INFO  (qtp359127126-18418) [n:127.0.0.1:48928_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2023641 INFO  (qtp359127126-18418) [n:127.0.0.1:48928_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2023642 INFO  (qtp359127126-18418) [n:127.0.0.1:48928_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@50b124aa[collection_two_shard1_replica_n1] main]
   [junit4]   2> 2023644 INFO  (qtp359127126-18418) [n:127.0.0.1:48928_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 2023644 INFO  (qtp359127126-18418) [n:127.0.0.1:48928_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 2023645 INFO  (qtp359127126-18418) [n:127.0.0.1:48928_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 2023645 INFO  (qtp359127126-18418) [n:127.0.0.1:48928_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1593841357891305472
   [junit4]   2> 2023645 INFO  (searcherExecutor-7256-thread-1-processing-n:127.0.0.1:48928_solr x:collection_two_shard1_replica_n1 s:shard1 c:collection_two r:core_node2) [n:127.0.0.1:48928_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.

[...truncated too long message...]

8 WARN  (Thread-3932) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	23	/solr/aliases.json
   [junit4]   2> 	23	/solr/clusterprops.json
   [junit4]   2> 	16	/solr/configs/conf
   [junit4]   2> 	9	/solr/collections/collection2meta/terms/shard1
   [junit4]   2> 	8	/solr/collections/collection1meta/terms/shard2
   [junit4]   2> 	8	/solr/collections/collection1meta/terms/shard1
   [junit4]   2> 	3	/solr/collections/collection_old/terms/shard1
   [junit4]   2> 	3	/solr/collections/collection1/terms/shard1
   [junit4]   2> 	2	/solr/collections/collection_new/terms/shard1
   [junit4]   2> 	2	/solr/collections/collection2/terms/shard1
   [junit4]   2> 	2	/solr/collections/collection_two/terms/shard1
   [junit4]   2> 	2	/solr/collections/collection_one/terms/shard1
   [junit4]   2> 	2	/solr/collections/collection_one/terms/shard2
   [junit4]   2> 	2	/solr/collections/collection_old/terms/shard2
   [junit4]   2> 	2	/solr/collections/testErrorChecks-collection/terms/shard1
   [junit4]   2> 	2	/solr/collections/testErrorChecks-collection/terms/shard2
   [junit4]   2> 	2	/solr/security.json
   [junit4]   2> 	2	/solr/collections/collection1/terms/shard2
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	216	/solr/aliases.json
   [junit4]   2> 	44	/solr/collections/collection1meta/state.json
   [junit4]   2> 	24	/solr/collections/collection2meta/state.json
   [junit4]   2> 	23	/solr/clusterstate.json
   [junit4]   2> 	11	/solr/collections/collection_one/state.json
   [junit4]   2> 	11	/solr/collections/collection1/state.json
   [junit4]   2> 	11	/solr/collections/collection_old/state.json
   [junit4]   2> 	11	/solr/collections/testErrorChecks-collection/state.json
   [junit4]   2> 	6	/solr/collections/collection_two/state.json
   [junit4]   2> 	6	/solr/collections/collection_new/state.json
   [junit4]   2> 	6	/solr/collections/collection2/state.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	204	/solr/collections
   [junit4]   2> 	23	/solr/live_nodes
   [junit4]   2> 
   [junit4]   2> NOTE: leaving temporary files on disk at: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.AliasIntegrationTest_E1F4D5F761FF3F81-001
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@7ff41422), locale=fi, timezone=SystemV/CST6
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_162 (64-bit)/cpus=3,threads=1,free=192326736,total=517799936
   [junit4]   2> NOTE: All tests run in this JVM: [TestTrackingShardHandlerFactory, TestConfigSetsAPI, TestSSLRandomization, SimplePostToolTest, ShowFileRequestHandlerTest, HttpTriggerListenerTest, InfixSuggestersTest, TestSimpleTrackingShardHandler, DateRangeFieldTest, RecoveryAfterSoftCommitTest, CdcrRequestHandlerTest, SolrXmlInZkTest, TestNumericRangeQuery64, DocumentAnalysisRequestHandlerTest, DeleteReplicaTest, TestJettySolrRunner, CloneFieldUpdateProcessorFactoryTest, HttpPartitionTest, TestFuzzyAnalyzedSuggestions, TestSolrConfigHandler, TestRetrieveFieldsOptimizer, LeaderFailureAfterFreshStartTest, TestUniqueKeyFieldResource, ConjunctionSolrSpellCheckerTest, SearchHandlerTest, TestClusterStateProvider, SolrCLIZkUtilsTest, TestSchemaVersionResource, TestAtomicUpdateErrorCases, TestCustomStream, OverseerTaskQueueTest, HttpSolrCallGetCoreTest, ClusterStateUpdateTest, TestCoreDiscovery, UpdateParamsTest, DistributedTermsComponentTest, TestLRUStatsCache, DistributedQueueTest, TestDocSet, TestFastLRUCache, DebugComponentTest, TestStressLucene, HdfsDirectoryTest, SuggesterWFSTTest, SolrMetricsIntegrationTest, TestRTGBase, TestShortCircuitedRequests, TestReqParamsAPI, SuggestComponentContextFilterQueryTest, CircularListTest, TestTriggerIntegration, DOMUtilTest, DocValuesTest, DistributedSuggestComponentTest, TestXmlQParserPlugin, TestObjectReleaseTracker, ShufflingReplicaListTransformerTest, NotRequiredUniqueKeyTest, ConcurrentCreateRoutedAliasTest, TestMacros, ReplicationFactorTest, WrapperMergePolicyFactoryTest, TestFieldResource, TimeZoneUtilsTest, HealthCheckHandlerTest, TestIndexingPerformance, TestSolr4Spatial, TestCrossCoreJoin, MultiThreadedOCPTest, TestExactSharedStatsCache, UUIDUpdateProcessorFallbackTest, CollectionTooManyReplicasTest, TestSolrCloudWithHadoopAuthPlugin, HdfsUnloadDistributedZkTest, TestSQLHandler, TestLegacyNumericRangeQueryBuilder, TestStandardQParsers, PrimUtilsTest, TestHdfsBackupRestoreCore, TestExtendedDismaxParser, ExternalFileFieldSortTest, AnalysisErrorHandlingTest, TestSlowCompositeReaderWrapper, TestAnalyzedSuggestions, TestJsonFacetRefinement, AsyncCallRequestStatusResponseTest, TestHttpShardHandlerFactory, JsonLoaderTest, MetricsConfigTest, TestStressReorder, TestCSVLoader, TestRandomFaceting, ReturnFieldsTest, TestRandomFlRTGCloud, TestDistribStateManager, SortByFunctionTest, TestSolrCloudWithKerberosAlt, DistributedIntervalFacetingTest, TestGenericDistributedQueue, ZkControllerTest, SolrCloudReportersTest, HDFSCollectionsAPITest, NoCacheHeaderTest, TestConfigSetImmutable, TestJsonRequest, TestOnReconnectListenerSupport, TestCloudJSONFacetJoinDomain, TestLMJelinekMercerSimilarityFactory, URLClassifyProcessorTest, DistributedQueryComponentCustomSortTest, TestDocumentBuilder, RegexBoostProcessorTest, BasicFunctionalityTest, TestRestManager, SolrCmdDistributorTest, DefaultValueUpdateProcessorTest, HdfsBasicDistributedZkTest, TestSolrQueryParser, PeerSyncTest, TestSchemalessBufferedUpdates, SolrJmxReporterTest, TestFastWriter, SystemInfoHandlerTest, SearchRateTriggerTest, HdfsRestartWhileUpdatingTest, TestSearchPerf, SliceStateTest, TestRebalanceLeaders, RequestLoggingTest, TestJavabinTupleStreamParser, TestLegacyNumericUtils, CreateRoutedAliasTest, TestEmbeddedSolrServerConstructors, SoftAutoCommitTest, PreAnalyzedUpdateProcessorTest, TestSolrCoreProperties, DeleteStatusTest, CoreAdminHandlerTest, TemplateUpdateProcessorTest, TestMultiValuedNumericRangeQuery, TestHighFrequencyDictionaryFactory, TestConfig, DistributedFacetPivotSmallTest, TestSort, TestCloudSearcherWarming, TestLegacyField, ZkFailoverTest, CdcrReplicationDistributedZkTest, TestAddFieldRealTimeGet, ActionThrottleTest, TestFieldTypeResource, TestFieldCacheWithThreads, DistributedDebugComponentTest, TestScoreJoinQPNoScore, RulesTest, ConfigureRecoveryStrategyTest, FullHLLTest, TestSearcherReuse, OverseerTest, TestQuerySenderListener, RequestHandlersTest, TestLeaderElectionZkExpiry, TestStressRecovery, CollectionStateFormat2Test, DirectUpdateHandlerTest, V2StandaloneTest, TestManagedSynonymFilterFactory, DocValuesNotIndexedTest, SimpleFacetsTest, XsltUpdateRequestHandlerTest, TestWriterPerf, DistribDocExpirationUpdateProcessorTest, OverseerCollectionConfigSetProcessorTest, TestValueSourceCache, FullSolrCloudDistribCmdsTest, OutOfBoxZkACLAndCredentialsProvidersTest, TestSolrCLIRunExample, TestDistributedMissingSort, TestRawResponseWriter, RegexBytesRefFilterTest, TestWordDelimiterFilterFactory, EnumFieldTest, TestChildDocTransformer, SpellCheckCollatorTest, TestDFRSimilarityFactory, RollingRestartTest, DistanceFunctionTest, PreAnalyzedFieldTest, TestDistributedMap, HighlighterConfigTest, TestDistributedGrouping, TestHashPartitioner, DistributedSpellCheckComponentTest, TestJoin, TestStressVersions, SolrCoreTest, TestGroupingSearch, BadIndexSchemaTest, TestFunctionQuery, TestBadConfig, TestIndexSearcher, DistributedQueryElevationComponentTest, SolrRequestParserTest, SuggesterTSTTest, SuggesterTest, SchemaVersionSpecificBehaviorTest, SolrCoreCheckLockOnStartupTest, TestUpdate, FieldMutatingUpdateProcessorTest, QueryEqualityTest, DocValuesMultiTest, IndexBasedSpellCheckerTest, TestReversedWildcardFilterFactory, PrimitiveFieldTypeTest, TestOmitPositions, XmlUpdateRequestHandlerTest, RequiredFieldsTest, FastVectorHighlighterTest, LoggingHandlerTest, UpdateRequestProcessorFactoryTest, MBeansHandlerTest, TestLFUCache, TestSolrDeletionPolicy2, MultiTermTest, TestBinaryField, TestElisionMultitermQuery, OutputWriterTest, TestPhraseSuggestions, SynonymTokenizerTest, TestXIncludeConfig, EchoParamsTest, TestIBSimilarityFactory, ResourceLoaderTest, OpenExchangeRatesOrgProviderTest, TestSuggestSpellingConverter, DateFieldTest, RAMDirectoryFactoryTest, ClusterStateTest, TestLRUCache, TestUtils, ZkNodePropsTest, UUIDFieldTest, FileUtilsTest, SolrTestCaseJ4Test, TestLuceneIndexBackCompat, TestEmbeddedSolrServerAdminHandler, ConnectionReuseTest, AliasIntegrationTest]
   [junit4] Completed [504/778 (1!)] on J1 in 66.89s, 8 tests, 1 error <<< FAILURES!

[...truncated 48729 lines...]
[repro] Jenkins log URL: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1708/consoleText

[repro] Revision: ca2131573de4c8d127ea80fdb2bd37e00c87bbcc

[repro] Ant options: "-Dargs=-XX:-UseCompressedOops -XX:+UseConcMarkSweepGC"
[repro] Repro line:  ant test  -Dtestcase=AliasIntegrationTest -Dtests.method=testModifyMetadataCAR -Dtests.seed=E1F4D5F761FF3F81 -Dtests.slow=true -Dtests.locale=fi -Dtests.timezone=SystemV/CST6 -Dtests.asserts=true -Dtests.file.encoding=US-ASCII

[repro] ant clean

[...truncated 6 lines...]
[repro] Test suites by module:
[repro]    solr/core
[repro]       AliasIntegrationTest
[repro] ant compile-test

[...truncated 3313 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 -Dtests.class="*.AliasIntegrationTest" -Dtests.showOutput=onerror "-Dargs=-XX:-UseCompressedOops -XX:+UseConcMarkSweepGC" -Dtests.seed=E1F4D5F761FF3F81 -Dtests.slow=true -Dtests.locale=fi -Dtests.timezone=SystemV/CST6 -Dtests.asserts=true -Dtests.file.encoding=US-ASCII

[...truncated 103 lines...]
[repro] Failures:
[repro]   0/5 failed: org.apache.solr.cloud.AliasIntegrationTest
[repro] Exiting with code 0

[...truncated 70 lines...]

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 1707 - Still Unstable!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1707/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseG1GC

2 tests failed.
FAILED:  org.apache.solr.cloud.AliasIntegrationTest.testMetadata

Error Message:
Error from server at http://127.0.0.1:42348/solr: Collection : collection2meta is part of alias meta3 remove or modify the alias before removing this collection.

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:42348/solr: Collection : collection2meta is part of alias meta3 remove or modify the alias before removing this collection.
	at __randomizedtesting.SeedInfo.seed([B7B563BE2D2C5BDF:BEA529BA6AC507B6]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
	at org.apache.solr.cloud.MiniSolrCloudCluster.deleteAllCollections(MiniSolrCloudCluster.java:444)
	at org.apache.solr.cloud.AliasIntegrationTest.tearDown(AliasIntegrationTest.java:92)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.AliasIntegrationTest.testModifyMetadataCAR

Error Message:
Error from server at http://127.0.0.1:42348/solr: collection already exists: collection1meta

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:42348/solr: collection already exists: collection1meta
	at __randomizedtesting.SeedInfo.seed([B7B563BE2D2C5BDF:ABE722F590F7FB87]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
	at org.apache.solr.cloud.AliasIntegrationTest.createColectionsAndAlias(AliasIntegrationTest.java:291)
	at org.apache.solr.cloud.AliasIntegrationTest.testModifyMetadataCAR(AliasIntegrationTest.java:257)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 12846 lines...]
   [junit4] Suite: org.apache.solr.cloud.AliasIntegrationTest
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.AliasIntegrationTest_B7B563BE2D2C5BDF-001/init-core-data-001
   [junit4]   2> 1599985 WARN  (SUITE-AliasIntegrationTest-seed#[B7B563BE2D2C5BDF]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
   [junit4]   2> 1599985 INFO  (SUITE-AliasIntegrationTest-seed#[B7B563BE2D2C5BDF]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 1599986 INFO  (SUITE-AliasIntegrationTest-seed#[B7B563BE2D2C5BDF]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 1599986 INFO  (SUITE-AliasIntegrationTest-seed#[B7B563BE2D2C5BDF]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 1599987 INFO  (SUITE-AliasIntegrationTest-seed#[B7B563BE2D2C5BDF]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.AliasIntegrationTest_B7B563BE2D2C5BDF-001/tempDir-001
   [junit4]   2> 1599987 INFO  (SUITE-AliasIntegrationTest-seed#[B7B563BE2D2C5BDF]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1599987 INFO  (Thread-3136) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1599987 INFO  (Thread-3136) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1599989 ERROR (Thread-3136) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1600087 INFO  (SUITE-AliasIntegrationTest-seed#[B7B563BE2D2C5BDF]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:33924
   [junit4]   2> 1600090 INFO  (zkConnectionManagerCallback-4137-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1600094 INFO  (jetty-launcher-4134-thread-1) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T18:27:37-03:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 1600094 INFO  (jetty-launcher-4134-thread-2) [    ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T18:27:37-03:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
   [junit4]   2> 1600094 INFO  (jetty-launcher-4134-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1600095 INFO  (jetty-launcher-4134-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1600095 INFO  (jetty-launcher-4134-thread-1) [    ] o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 1600095 INFO  (jetty-launcher-4134-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@242e6368{/solr,null,AVAILABLE}
   [junit4]   2> 1600095 INFO  (jetty-launcher-4134-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1600095 INFO  (jetty-launcher-4134-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1600095 INFO  (jetty-launcher-4134-thread-2) [    ] o.e.j.s.session Scavenging every 660000ms
   [junit4]   2> 1600096 INFO  (jetty-launcher-4134-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@42f74635{HTTP/1.1,[http/1.1]}{127.0.0.1:42348}
   [junit4]   2> 1600096 INFO  (jetty-launcher-4134-thread-1) [    ] o.e.j.s.Server Started @1602109ms
   [junit4]   2> 1600096 INFO  (jetty-launcher-4134-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42348}
   [junit4]   2> 1600096 INFO  (jetty-launcher-4134-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@33c17b1e{/solr,null,AVAILABLE}
   [junit4]   2> 1600096 ERROR (jetty-launcher-4134-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1600096 INFO  (jetty-launcher-4134-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@6dba0be8{HTTP/1.1,[http/1.1]}{127.0.0.1:38726}
   [junit4]   2> 1600096 INFO  (jetty-launcher-4134-thread-2) [    ] o.e.j.s.Server Started @1602110ms
   [junit4]   2> 1600096 INFO  (jetty-launcher-4134-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 1600096 INFO  (jetty-launcher-4134-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1600096 INFO  (jetty-launcher-4134-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1600096 INFO  (jetty-launcher-4134-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-03-02T04:06:32.975Z
   [junit4]   2> 1600096 INFO  (jetty-launcher-4134-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=38726}
   [junit4]   2> 1600096 ERROR (jetty-launcher-4134-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1600096 INFO  (jetty-launcher-4134-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.0.0
   [junit4]   2> 1600096 INFO  (jetty-launcher-4134-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1600096 INFO  (jetty-launcher-4134-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1600096 INFO  (jetty-launcher-4134-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-03-02T04:06:32.975Z
   [junit4]   2> 1600098 INFO  (zkConnectionManagerCallback-4141-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1600099 INFO  (zkConnectionManagerCallback-4139-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1600099 INFO  (jetty-launcher-4134-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1600100 INFO  (jetty-launcher-4134-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1600106 INFO  (jetty-launcher-4134-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33924/solr
   [junit4]   2> 1600107 INFO  (jetty-launcher-4134-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33924/solr
   [junit4]   2> 1600124 INFO  (zkConnectionManagerCallback-4147-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1600125 INFO  (zkConnectionManagerCallback-4149-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1600127 INFO  (zkConnectionManagerCallback-4153-thread-1-processing-n:127.0.0.1:42348_solr) [n:127.0.0.1:42348_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1600127 INFO  (zkConnectionManagerCallback-4151-thread-1-processing-n:127.0.0.1:38726_solr) [n:127.0.0.1:38726_solr    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1600285 INFO  (jetty-launcher-4134-thread-1) [n:127.0.0.1:42348_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1600287 INFO  (jetty-launcher-4134-thread-1) [n:127.0.0.1:42348_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42348_solr
   [junit4]   2> 1600289 INFO  (jetty-launcher-4134-thread-1) [n:127.0.0.1:42348_solr    ] o.a.s.c.Overseer Overseer (id=72064196424105990-127.0.0.1:42348_solr-n_0000000000) starting
   [junit4]   2> 1600300 INFO  (jetty-launcher-4134-thread-1) [n:127.0.0.1:42348_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42348_solr
   [junit4]   2> 1600305 INFO  (OverseerStateUpdate-72064196424105990-127.0.0.1:42348_solr-n_0000000000) [n:127.0.0.1:42348_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1600319 INFO  (jetty-launcher-4134-thread-2) [n:127.0.0.1:38726_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1600324 INFO  (jetty-launcher-4134-thread-2) [n:127.0.0.1:38726_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1600326 INFO  (jetty-launcher-4134-thread-2) [n:127.0.0.1:38726_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38726_solr
   [junit4]   2> 1600330 INFO  (zkCallback-4152-thread-1-processing-n:127.0.0.1:42348_solr) [n:127.0.0.1:42348_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1600336 INFO  (zkCallback-4150-thread-1-processing-n:127.0.0.1:38726_solr) [n:127.0.0.1:38726_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1600491 INFO  (jetty-launcher-4134-thread-1) [n:127.0.0.1:42348_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42348.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@12d44a87
   [junit4]   2> 1600499 INFO  (jetty-launcher-4134-thread-2) [n:127.0.0.1:38726_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38726.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@12d44a87
   [junit4]   2> 1600500 INFO  (jetty-launcher-4134-thread-1) [n:127.0.0.1:42348_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42348.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@12d44a87
   [junit4]   2> 1600500 INFO  (jetty-launcher-4134-thread-1) [n:127.0.0.1:42348_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42348.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@12d44a87
   [junit4]   2> 1600501 INFO  (jetty-launcher-4134-thread-1) [n:127.0.0.1:42348_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.AliasIntegrationTest_B7B563BE2D2C5BDF-001/tempDir-001/node1/.
   [junit4]   2> 1600505 INFO  (jetty-launcher-4134-thread-2) [n:127.0.0.1:38726_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38726.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@12d44a87
   [junit4]   2> 1600505 INFO  (jetty-launcher-4134-thread-2) [n:127.0.0.1:38726_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38726.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@12d44a87
   [junit4]   2> 1600506 INFO  (jetty-launcher-4134-thread-2) [n:127.0.0.1:38726_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.AliasIntegrationTest_B7B563BE2D2C5BDF-001/tempDir-001/node2/.
   [junit4]   2> 1600543 INFO  (zkConnectionManagerCallback-4159-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1600547 INFO  (zkConnectionManagerCallback-4163-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1600548 INFO  (SUITE-AliasIntegrationTest-seed#[B7B563BE2D2C5BDF]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 1600550 INFO  (SUITE-AliasIntegrationTest-seed#[B7B563BE2D2C5BDF]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33924/solr ready
   [junit4]   2> 1600585 INFO  (TEST-AliasIntegrationTest.testDeleteOneOfTwoCollectionsAliased-seed#[B7B563BE2D2C5BDF]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testDeleteOneOfTwoCollectionsAliased
   [junit4]   2> 1600589 INFO  (qtp1990662100-14479) [n:127.0.0.1:38726_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :clusterstatus with params action=CLUSTERSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1600591 INFO  (qtp1990662100-14479) [n:127.0.0.1:38726_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=CLUSTERSTATUS&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 1600594 INFO  (qtp207927362-14476) [n:127.0.0.1:42348_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf&name=collection_one&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1600597 INFO  (OverseerThreadFactory-4867-thread-1-processing-n:127.0.0.1:42348_solr) [n:127.0.0.1:42348_solr    ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection_one
   [junit4]   2> 1600708 INFO  (OverseerStateUpdate-72064196424105990-127.0.0.1:42348_solr-n_0000000000) [n:127.0.0.1:42348_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"collection_one",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"collection_one_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:42348/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 1600711 INFO  (OverseerStateUpdate-72064196424105990-127.0.0.1:42348_solr-n_0000000000) [n:127.0.0.1:42348_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"collection_one",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"collection_one_shard2_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:38726/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 1600914 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=collection_one_shard1_replica_n1&action=CREATE&numShards=2&collection=collection_one&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1600914 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1600916 INFO  (qtp1990662100-14489) [n:127.0.0.1:38726_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=collection_one_shard2_replica_n2&action=CREATE&numShards=2&collection=collection_one&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1600916 INFO  (qtp1990662100-14489) [n:127.0.0.1:38726_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1601022 INFO  (zkCallback-4150-thread-1-processing-n:127.0.0.1:38726_solr) [n:127.0.0.1:38726_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_one/state.json] for collection [collection_one] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1601022 INFO  (zkCallback-4152-thread-1-processing-n:127.0.0.1:42348_solr) [n:127.0.0.1:42348_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_one/state.json] for collection [collection_one] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1601022 INFO  (zkCallback-4152-thread-2-processing-n:127.0.0.1:42348_solr) [n:127.0.0.1:42348_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_one/state.json] for collection [collection_one] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1601022 INFO  (zkCallback-4150-thread-2-processing-n:127.0.0.1:38726_solr) [n:127.0.0.1:38726_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_one/state.json] for collection [collection_one] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1601965 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1601970 INFO  (qtp1990662100-14489) [n:127.0.0.1:38726_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1602011 INFO  (qtp1990662100-14489) [n:127.0.0.1:38726_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.s.IndexSchema [collection_one_shard2_replica_n2] Schema name=minimal
   [junit4]   2> 1602017 INFO  (qtp1990662100-14489) [n:127.0.0.1:38726_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1602017 INFO  (qtp1990662100-14489) [n:127.0.0.1:38726_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'collection_one_shard2_replica_n2' using configuration from collection collection_one, trusted=true
   [junit4]   2> 1602017 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.s.IndexSchema [collection_one_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 1602017 INFO  (qtp1990662100-14489) [n:127.0.0.1:38726_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38726.solr.core.collection_one.shard2.replica_n2' (registry 'solr.core.collection_one.shard2.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@12d44a87
   [junit4]   2> 1602017 INFO  (qtp1990662100-14489) [n:127.0.0.1:38726_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1602017 INFO  (qtp1990662100-14489) [n:127.0.0.1:38726_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.c.SolrCore [[collection_one_shard2_replica_n2] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.AliasIntegrationTest_B7B563BE2D2C5BDF-001/tempDir-001/node2/collection_one_shard2_replica_n2], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.AliasIntegrationTest_B7B563BE2D2C5BDF-001/tempDir-001/node2/./collection_one_shard2_replica_n2/data/]
   [junit4]   2> 1602020 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1602020 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'collection_one_shard1_replica_n1' using configuration from collection collection_one, trusted=true
   [junit4]   2> 1602021 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42348.solr.core.collection_one.shard1.replica_n1' (registry 'solr.core.collection_one.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@12d44a87
   [junit4]   2> 1602021 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1602021 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.c.SolrCore [[collection_one_shard1_replica_n1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.AliasIntegrationTest_B7B563BE2D2C5BDF-001/tempDir-001/node1/collection_one_shard1_replica_n1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.AliasIntegrationTest_B7B563BE2D2C5BDF-001/tempDir-001/node1/./collection_one_shard1_replica_n1/data/]
   [junit4]   2> 1602428 INFO  (qtp1990662100-14489) [n:127.0.0.1:38726_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1602428 INFO  (qtp1990662100-14489) [n:127.0.0.1:38726_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1602430 INFO  (qtp1990662100-14489) [n:127.0.0.1:38726_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1602430 INFO  (qtp1990662100-14489) [n:127.0.0.1:38726_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1602431 INFO  (qtp1990662100-14489) [n:127.0.0.1:38726_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@42c218e2[collection_one_shard2_replica_n2] main]
   [junit4]   2> 1602432 INFO  (qtp1990662100-14489) [n:127.0.0.1:38726_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 1602433 INFO  (qtp1990662100-14489) [n:127.0.0.1:38726_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 1602433 INFO  (qtp1990662100-14489) [n:127.0.0.1:38726_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1602434 INFO  (qtp1990662100-14489) [n:127.0.0.1:38726_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1593797346918924288
   [junit4]   2> 1602436 INFO  (searcherExecutor-4872-thread-1-processing-n:127.0.0.1:38726_solr x:collection_one_shard2_replica_n2 s:shard2 c:collection_one r:core_node4) [n:127.0.0.1:38726_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.c.SolrCore [collection_one_shard2_replica_n2] Registered new searcher Searcher@42c218e2[collection_one_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1602438 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1602438 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1602444 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1602444 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1602446 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2e109c3f[collection_one_shard1_replica_n1] main]
   [junit4]   2> 1602450 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 1602451 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 1602452 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1602452 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1593797346937798656
   [junit4]   2> 1602453 INFO  (searcherExecutor-4873-thread-1-processing-n:127.0.0.1:42348_solr x:collection_one_shard1_replica_n1 s:shard1 c:collection_one r:core_node3) [n:127.0.0.1:42348_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.c.SolrCore [collection_one_shard1_replica_n1] Registered new searcher Searcher@2e109c3f[collection_one_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1602480 INFO  (qtp1990662100-14489) [n:127.0.0.1:38726_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1602481 INFO  (qtp1990662100-14489) [n:127.0.0.1:38726_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1602481 INFO  (qtp1990662100-14489) [n:127.0.0.1:38726_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:38726/solr/collection_one_shard2_replica_n2/
   [junit4]   2> 1602481 INFO  (qtp1990662100-14489) [n:127.0.0.1:38726_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1602481 INFO  (qtp1990662100-14489) [n:127.0.0.1:38726_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.c.SyncStrategy http://127.0.0.1:38726/solr/collection_one_shard2_replica_n2/ has no replicas
   [junit4]   2> 1602481 INFO  (qtp1990662100-14489) [n:127.0.0.1:38726_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 1602488 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1602488 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1602488 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:42348/solr/collection_one_shard1_replica_n1/
   [junit4]   2> 1602488 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1602488 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:42348/solr/collection_one_shard1_replica_n1/ has no replicas
   [junit4]   2> 1602488 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 1602489 INFO  (qtp1990662100-14489) [n:127.0.0.1:38726_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:38726/solr/collection_one_shard2_replica_n2/ shard2
   [junit4]   2> 1602495 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:42348/solr/collection_one_shard1_replica_n1/ shard1
   [junit4]   2> 1602597 INFO  (zkCallback-4152-thread-1-processing-n:127.0.0.1:42348_solr) [n:127.0.0.1:42348_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_one/state.json] for collection [collection_one] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1602597 INFO  (zkCallback-4150-thread-2-processing-n:127.0.0.1:38726_solr) [n:127.0.0.1:38726_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_one/state.json] for collection [collection_one] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1602597 INFO  (zkCallback-4150-thread-1-processing-n:127.0.0.1:38726_solr) [n:127.0.0.1:38726_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_one/state.json] for collection [collection_one] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1602597 INFO  (zkCallback-4152-thread-2-processing-n:127.0.0.1:42348_solr) [n:127.0.0.1:42348_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_one/state.json] for collection [collection_one] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1602641 INFO  (qtp1990662100-14489) [n:127.0.0.1:38726_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1602643 INFO  (qtp1990662100-14489) [n:127.0.0.1:38726_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=collection_one_shard2_replica_n2&action=CREATE&numShards=2&collection=collection_one&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1727
   [junit4]   2> 1602646 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1602648 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=collection_one_shard1_replica_n1&action=CREATE&numShards=2&collection=collection_one&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1734
   [junit4]   2> 1602650 INFO  (qtp207927362-14476) [n:127.0.0.1:42348_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 1602749 INFO  (zkCallback-4150-thread-1-processing-n:127.0.0.1:38726_solr) [n:127.0.0.1:38726_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_one/state.json] for collection [collection_one] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1602749 INFO  (zkCallback-4152-thread-3-processing-n:127.0.0.1:42348_solr) [n:127.0.0.1:42348_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_one/state.json] for collection [collection_one] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1602749 INFO  (zkCallback-4152-thread-2-processing-n:127.0.0.1:42348_solr) [n:127.0.0.1:42348_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_one/state.json] for collection [collection_one] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1602749 INFO  (zkCallback-4150-thread-2-processing-n:127.0.0.1:38726_solr) [n:127.0.0.1:38726_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_one/state.json] for collection [collection_one] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1603652 INFO  (qtp207927362-14476) [n:127.0.0.1:42348_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf&name=collection_one&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=3058
   [junit4]   2> 1603652 INFO  (qtp207927362-14478) [n:127.0.0.1:42348_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf&name=collection_two&nrtReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1603655 INFO  (OverseerCollectionConfigSetProcessor-72064196424105990-127.0.0.1:42348_solr-n_0000000000) [n:127.0.0.1:42348_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1603656 INFO  (OverseerThreadFactory-4867-thread-2-processing-n:127.0.0.1:42348_solr) [n:127.0.0.1:42348_solr    ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection_two
   [junit4]   2> 1603867 INFO  (OverseerStateUpdate-72064196424105990-127.0.0.1:42348_solr-n_0000000000) [n:127.0.0.1:42348_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"collection_two",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"collection_two_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:42348/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 1604069 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr    ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf&newCollection=true&name=collection_two_shard1_replica_n1&action=CREATE&numShards=1&collection=collection_two&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1604174 INFO  (zkCallback-4152-thread-3-processing-n:127.0.0.1:42348_solr) [n:127.0.0.1:42348_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_two/state.json] for collection [collection_two] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1604174 INFO  (zkCallback-4152-thread-2-processing-n:127.0.0.1:42348_solr) [n:127.0.0.1:42348_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_two/state.json] for collection [collection_two] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1605082 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1605087 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.s.IndexSchema [collection_two_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 1605089 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1605089 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'collection_two_shard1_replica_n1' using configuration from collection collection_two, trusted=true
   [junit4]   2> 1605089 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42348.solr.core.collection_two.shard1.replica_n1' (registry 'solr.core.collection_two.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@12d44a87
   [junit4]   2> 1605089 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1605089 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.c.SolrCore [[collection_two_shard1_replica_n1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.AliasIntegrationTest_B7B563BE2D2C5BDF-001/tempDir-001/node1/collection_two_shard1_replica_n1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.AliasIntegrationTest_B7B563BE2D2C5BDF-001/tempDir-001/node1/./collection_two_shard1_replica_n1/data/]
   [junit4]   2> 1605149 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1605149 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1605229 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1605229 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1605232 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3cf5ee53[collection_two_shard1_replica_n1] main]
   [junit4]   2> 1605234 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 1605234 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 1605235 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1605235 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1593797349855985664
   [junit4]   2> 1605236 INFO  (searcherExecutor-4882-thread-1-processing-n:127.0.0.1:42348_solr x:collection_two_shard1_replica_n1 s:shard1 c:collection_two r:core_node2) [n:127.0.0.1:42348_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.c.SolrCore [collection_two_shard1_replica_n1] Registered new searcher Searcher@3cf5ee53[collection_two_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1605248 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1605248 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1605248 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:42348/solr/collection_two_shard1_replica_n1/
   [junit4]   2> 1605248 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1605248 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:42348/solr/collection_two_shard1_replica_n1/ has no replicas
   [junit4]   2> 1605248 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 1605253 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:42348/solr/collection_two_shard1_replica_n1/ shard1
   [junit4]   2> 1605355 INFO  (zkCallback-4152-thread-1-processing-n:127.0.0.1:42348_solr) [n:127.0.0.1:42348_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_two/state.json] for collection [collection_two] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1605355 INFO  (zkCallback-4152-thread-2-processing-n:127.0.0.1:42348_solr) [n:127.0.0.1:42348_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_two/state.json] for collection [collection_two] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1605404 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1605406 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf&newCollection=true&name=collection_two_shard1_replica_n1&action=CREATE&numShards=1&collection=collection_two&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1337
   [junit4]   2> 1605408 INFO  (qtp207927362-14478) [n:127.0.0.1:42348_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 1605508 INFO  (zkCallback-4152-thread-1-processing-n:127.0.0.1:42348_solr) [n:127.0.0.1:42348_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_two/state.json] for collection [collection_two] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1605508 INFO  (zkCallback-4152-thread-2-processing-n:127.0.0.1:42348_solr) [n:127.0.0.1:42348_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection_two/state.json] for collection [collection_two] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1605657 INFO  (OverseerCollectionConfigSetProcessor-72064196424105990-127.0.0.1:42348_solr-n_0000000000) [n:127.0.0.1:42348_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1606410 INFO  (qtp207927362-14478) [n:127.0.0.1:42348_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf&name=collection_two&nrtReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=2757
   [junit4]   2> 1606414 INFO  (qtp207927362-14482) [n:127.0.0.1:42348_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection_one_shard1_replica_n1]  webapp=/solr path=/update params={_stateVer_=collection_one:4&wt=javabin&version=2}{add=[1 (1593797351091208192)]} 0 1
   [junit4]   2> 1606417 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1593797351095402496,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1606417 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@3022f721 commitCommandVersion:1593797351095402496
   [junit4]   2> 1606417 INFO  (qtp1990662100-14489) [n:127.0.0.1:38726_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1593797351095402496,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1606417 INFO  (qtp1990662100-14489) [n:127.0.0.1:38726_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 1606417 INFO  (qtp1990662100-14489) [n:127.0.0.1:38726_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1606417 INFO  (qtp1990662100-14489) [n:127.0.0.1:38726_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [collection_one_shard2_replica_n2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:42348/solr/collection_one_shard1_replica_n1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 0
   [junit4]   2> 1606418 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@531db9ea[collection_one_shard1_replica_n1] main]
   [junit4]   2> 1606420 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1606421 INFO  (searcherExecutor-4873-thread-1-processing-n:127.0.0.1:42348_solr x:collection_one_shard1_replica_n1 s:shard1 c:collection_one r:core_node3) [n:127.0.0.1:42348_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.c.SolrCore [collection_one_shard1_replica_n1] Registered new searcher Searcher@531db9ea[collection_one_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(8.0.0):C1)))}
   [junit4]   2> 1606421 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection_one_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:42348/solr/collection_one_shard1_replica_n1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 4
   [junit4]   2> 1606421 INFO  (qtp207927362-14478) [n:127.0.0.1:42348_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection_one_shard1_replica_n1]  webapp=/solr path=/update params={_stateVer_=collection_one:4&commit=true&wt=javabin&version=2}{commit=} 0 6
   [junit4]   2> 1606426 INFO  (qtp207927362-14482) [n:127.0.0.1:42348_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection_two_shard1_replica_n1]  webapp=/solr path=/update params={_stateVer_=collection_two:4&wt=javabin&version=2}{add=[10 (1593797351101693952), 11 (1593797351104839680)]} 0 2
   [junit4]   2> 1606427 INFO  (qtp207927362-14478) [n:127.0.0.1:42348_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1593797351105888256,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1606427 INFO  (qtp207927362-14478) [n:127.0.0.1:42348_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@3c2ba454 commitCommandVersion:1593797351105888256
   [junit4]   2> 1606430 INFO  (qtp207927362-14478) [n:127.0.0.1:42348_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@210b84bd[collection_two_shard1_replica_n1] main]
   [junit4]   2> 1606431 INFO  (qtp207927362-14478) [n:127.0.0.1:42348_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1606433 INFO  (searcherExecutor-4882-thread-1-processing-n:127.0.0.1:42348_solr x:collection_two_shard1_replica_n1 s:shard1 c:collection_two r:core_node2) [n:127.0.0.1:42348_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.c.SolrCore [collection_two_shard1_replica_n1] Registered new searcher Searcher@210b84bd[collection_two_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(8.0.0):C2)))}
   [junit4]   2> 1606433 INFO  (qtp207927362-14478) [n:127.0.0.1:42348_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection_two_shard1_replica_n1]  webapp=/solr path=/update params={_stateVer_=collection_two:4&commit=true&wt=javabin&version=2}{commit=} 0 5
   [junit4]   2> 1606438 INFO  (qtp207927362-14482) [n:127.0.0.1:42348_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createalias with params collections=collection_one,collection_two&name=collection_alias_pair&action=CREATEALIAS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1606545 INFO  (qtp207927362-14482) [n:127.0.0.1:42348_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collections=collection_one,collection_two&name=collection_alias_pair&action=CREATEALIAS&wt=javabin&version=2} status=0 QTime=107
   [junit4]   2> 1606547 INFO  (qtp207927362-14482) [n:127.0.0.1:42348_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.c.S.Request [collection_one_shard1_replica_n1]  webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=collection_one:4|collection_two:4&fl=id&fl=score&shards.purpose=4&start=0&fsv=true&collection=collection_one,collection_two&shard.url=http://127.0.0.1:42348/solr/collection_one_shard1_replica_n1/&rows=10&version=2&q=*:*&NOW=1519963599425&isShard=true&wt=javabin} hits=1 status=0 QTime=0
   [junit4]   2> 1606548 INFO  (qtp207927362-14482) [n:127.0.0.1:42348_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.c.S.Request [collection_two_shard1_replica_n1]  webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=collection_one:4|collection_two:4&fl=id&fl=score&shards.purpose=4&start=0&fsv=true&collection=collection_one,collection_two&shard.url=http://127.0.0.1:42348/solr/collection_two_shard1_replica_n1/&rows=10&version=2&q=*:*&NOW=1519963599425&isShard=true&wt=javabin} hits=2 status=0 QTime=0
   [junit4]   2> 1606551 INFO  (qtp1990662100-14486) [n:127.0.0.1:38726_solr c:collection_one s:shard2 r:core_node4 x:collection_one_shard2_replica_n2] o.a.s.c.S.Request [collection_one_shard2_replica_n2]  webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=collection_one:4|collection_two:4&fl=id&fl=score&shards.purpose=4&start=0&fsv=true&collection=collection_one,collection_two&shard.url=http://127.0.0.1:38726/solr/collection_one_shard2_replica_n2/&rows=10&version=2&q=*:*&NOW=1519963599425&isShard=true&wt=javabin} hits=0 status=0 QTime=2
   [junit4]   2> 1606553 INFO  (qtp207927362-14481) [n:127.0.0.1:42348_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.c.S.Request [collection_one_shard1_replica_n1]  webapp=/solr path=/select params={q=*:*&df=text&distrib=false&_stateVer_=collection_one:4|collection_two:4&shards.purpose=64&NOW=1519963599425&ids=1&isShard=true&collection=collection_one,collection_two&shard.url=http://127.0.0.1:42348/solr/collection_one_shard1_replica_n1/&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1606553 INFO  (qtp207927362-14482) [n:127.0.0.1:42348_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.c.S.Request [collection_two_shard1_replica_n1]  webapp=/solr path=/select params={q=*:*&df=text&distrib=false&_stateVer_=collection_one:4|collection_two:4&shards.purpose=64&NOW=1519963599425&ids=11,10&isShard=true&collection=collection_one,collection_two&shard.url=http://127.0.0.1:42348/solr/collection_two_shard1_replica_n1/&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1606553 INFO  (qtp207927362-14478) [n:127.0.0.1:42348_solr c:collection_one s:shard1 r:core_node3 x:collection_one_shard1_replica_n1] o.a.s.c.S.Request [collection_one_shard1_replica_n1]  webapp=/solr path=/select params={q=*:*&_stateVer_=collection_one:4|collection_two:4&wt=javabin&version=2} hits=3 status=0 QTime=6
   [junit4]   2> 1606554 INFO  (qtp207927362-14483) [n:127.0.0.1:42348_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params async=7fa4bcb7-a0e1-4651-a4f6-ddf338c1bb52&name=collection_one&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1606557 INFO  (qtp207927362-14483) [n:127.0.0.1:42348_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=7fa4bcb7-a0e1-4651-a4f6-ddf338c1bb52&name=collection_one&action=DELETE&wt=javabin&version=2} status=0 QTime=3
   [junit4]   2> 1606558 INFO  (qtp207927362-14485) [n:127.0.0.1:42348_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=7fa4bcb7-a0e1-4651-a4f6-ddf338c1bb52&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1606560 INFO  (qtp207927362-14485) [n:127.0.0.1:42348_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=7fa4bcb7-a0e1-4651-a4f6-ddf338c1bb52&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2
   [junit4]   2> 1606561 ERROR (OverseerThreadFactory-4867-thread-4-processing-n:127.0.0.1:42348_solr) [n:127.0.0.1:42348_solr    ] o.a.s.c.a.c.OverseerCollectionMessageHandler Collection: collection_one operation: delete failed:org.apache.solr.common.SolrException: Collection : collection_one is part of alias collection_alias_pair remove or modify the alias before removing this collection.
   [junit4]   2> 	at org.apache.solr.cloud.api.collections.DeleteCollectionCmd.call(DeleteCollectionCmd.java:71)
   [junit4]   2> 	at org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:253)
   [junit4]   2> 	at org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:469)
   [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 
   [junit4]   2> 1606562 INFO  (OverseerCollectionConfigSetProcessor-72064196424105990-127.0.0.1:42348_solr-n_0000000000) [n:127.0.0.1:42348_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1606564 INFO  (OverseerThreadFactory-4867-thread-4-processing-n:127.0.0.1:42348_solr) [n:127.0.0.1:42348_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1607561 INFO  (qtp207927362-14483) [n:127.0.0.1:42348_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=7fa4bcb7-a0e1-4651-a4f6-ddf338c1bb52&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1607563 INFO  (qtp207927362-14483) [n:127.0.0.1:42348_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=7fa4bcb7-a0e1-4651-a4f6-ddf338c1bb52&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 1607564 INFO  (qtp207927362-14485) [n:127.0.0.1:42348_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :deletestatus with params requestid=7fa4bcb7-a0e1-4651-a4f6-ddf338c1bb52&action=DELETESTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1607565 INFO  (qtp207927362-14485) [n:127.0.0.1:42348_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=7fa4bcb7-a0e1-4651-a4f6-ddf338c1bb52&action=DELETESTATUS&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 1607565 INFO  (qtp207927362-14483) [n:127.0.0.1:42348_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createalias with params collections=collection_two&name=collection_alias_pair&action=CREATEALIAS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1607568 INFO  (OverseerCollectionConfigSetProcessor-72064196424105990-127.0.0.1:42348_solr-n_0000000000) [n:127.0.0.1:42348_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1607671 INFO  (qtp207927362-14483) [n:127.0.0.1:42348_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collections=collection_two&name=collection_alias_pair&action=CREATEALIAS&wt=javabin&version=2} status=0 QTime=105
   [junit4]   2> 1607671 INFO  (qtp207927362-14485) [n:127.0.0.1:42348_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params async=db6777c4-94cc-4096-898c-3dd9b853a18c&name=collection_one&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1607674 INFO  (qtp207927362-14485) [n:127.0.0.1:42348_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=db6777c4-94cc-4096-898c-3dd9b853a18c&name=collection_one&action=DELETE&wt=javabin&version=2} status=0 QTime=3
   [junit4]   2> 1607675 INFO  (qtp207927362-14483) [n:127.0.0.1:42348_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=db6777c4-94cc-4096-898c-3dd9b853a18c&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1607676 INFO  (qtp207927362-14483) [n:127.0.0.1:42348_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=db6777c4-94cc-4096-898c-3dd9b853a18c&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 1607677 INFO  (OverseerThreadFactory-4867-thread-5-processing-n:127.0.0.1:42348_solr) [n:127.0.0.1:42348_solr    ] o.a.s.c.a.c.OverseerCollectionMessageHandler Executing Collection Cmd=action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true, asyncId=db6777c4-94cc-4096-898c-3dd9b853a18c
   [junit4]   2> 1607678 INFO  (OverseerCollectionConfigSetProcessor-72064196424105990-127.0.0.1:42348_solr-n_0000000000) [n:127.0.0.1:42348_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000007 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1607678 INFO  (qtp1990662100-14489) [n:127.0.0.1:38726_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=collection_one_shard2_replica_n2&async=db6777c4-94cc-4096-898c-3dd9b853a18c100752109001594&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1607678 INFO  (qtp207927362-14485) [n:127.0.0.1:42348_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=collection_one_shard1_replica_n1&async=db6777c4-94cc-4096-898c-3dd9b853a18c100752108950591&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1607679 INFO  (qtp207927362-14483) [n:127.0.0.1:42348_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : db6777c4-94cc-4096-898c-3dd9b853a18c100752108950591
   [junit4]   2> 1607679 INFO  (qtp207927362-14483) [n:127.0.0.1:42348_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=db6777c4-94cc-4096-898c-3dd9b853a18c100752108950591&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1607679 INFO  (parallelCoreAdminExecutor-4868-thread-1-processing-n:127.0.0.1:42348_solr db6777c4-94cc-4096-898c-3dd9b853a18c100752108950591 UNLOAD) [n:127.0.0.1:42348_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection_one.shard1.replica_n1, tag=null
   [junit4]   2> 1607679 INFO  (parallelCoreAdminExecutor-4869-thread-1-processing-n:127.0.0.1:38726_solr db6777c4-94cc-4096-898c-3dd9b853a18c100752109001594 UNLOAD) [n:127.0.0.1:38726_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection_one.shard2.replica_n2, tag=null
   [junit4]   2> 1607679 INFO  (parallelCoreAdminExecutor-4868-thread-1-processing-n:127.0.0.1:42348_solr db6777c4-94cc-4096-898c-3dd9b853a18c100752108950591 UNLOAD) [n:127.0.0.1:42348_solr    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@685c29d3: rootName = solr_42348, domain = solr.core.collection_one.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.collection_one.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@1ddf7109
   [junit4]   2> 1607679 INFO  (parallelCoreAdminExecutor-4869-thread-1-processing-n:127.0.0.1:38726_solr db6777c4-94cc-4096-898c-3dd9b853a18c100752109001594 UNLOAD) [n:127.0.0.1:38726_solr    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3b76ad2a: rootName = solr_38726, domain = solr.core.collection_one.shard2.replica_n2, service url = null, agent id = null] for registry solr.core.collection_one.shard2.replica_n2 / com.codahale.metrics.MetricRegistry@5bb026d3
   [junit4]   2> 1607715 INFO  (parallelCoreAdminExecutor-4868-thread-1-processing-n:127.0.0.1:42348_solr db6777c4-94cc-4096-898c-3dd9b853a18c100752108950591 UNLOAD) [n:127.0.0.1:42348_solr    ] o.a.s.c.SolrCore [collection_one_shard1_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@706a403d
   [junit4]   2> 1607715 INFO  (parallelCoreAdminExecutor-4868-thread-1-processing-n:127.0.0.1:42348_solr db6777c4-94cc-4096-898c-3dd9b853a18c100752108950591 UNLOAD) [n:127.0.0.1:42348_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection_one.shard1.replica_n1, tag=1886011453
   [junit4]   2> 1607715 INFO  (parallelCoreAdminExecutor-4868-thread-1-processing-n:127.0.0.1:42348_solr db6777c4-94cc-4096-898c-3dd9b853a18c100752108950591 UNLOAD) [n:127.0.0.1:42348_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection_one.shard1.leader, tag=1886011453
   [junit4]   2> 1607715 INFO  (parallelCoreAdminExecutor-4869-thread-1-processing-n:127.0.0.1:38726_solr db6777c4-94cc-4096-898c-3dd9b853a18c100752109001594 UNLOAD) [n:127.0.0.1:38726_solr    ] o.a.s.c.SolrCore [collection_one_shard2_replica_n2]  CLOSING SolrCore org.apache.solr.core.SolrCore@6d7d234c
   [junit4]   2> 1607715 INFO  (parallelCoreAdminExecutor-4869-thread-1-processing-n:127.0.0.1:38726_solr db6777c4-94cc-4096-898c-3dd9b853a18c100752109001594 UNLOAD) [n:127.0.0.1:38726_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection_one.shard2.replica_n2, tag=1836917580
   [junit4]   2> 1607715 INFO  (parallelCoreAdminExecutor-4869-thread-1-processing-n:127.0.0.1:38726_solr db6777c4-94cc-4096-898c-3dd9b853a18c100752109001594 UNLOAD) [n:127.0.0.1:38726_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection_one.shard2.leader, tag=1836917580
   [junit4]   2> 1608681 INFO  (qtp207927362-14485) [n:127.0.0.1:42348_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=db6777c4-94cc-4096-898c-3dd9b853a18c&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1608681 INFO  (qtp207927362-14483) [n:127.0.0.1:42348_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : db6777c4-94cc-4096-898c-3dd9b853a18c100752108950591
   [junit4]   2> 1608681 INFO  (qtp207927362-14483) [n:127.0.0.1:42348_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=db6777c4-94cc-4096-898c-3dd9b853a18c100752108950591&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1608681 INFO  (qtp1990662100-14486) [n:127.0.0.1:38726_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : db6777c4-94cc-4096-898c-3dd9b853a18c100752109001594
   [junit4]   2> 1608681 INFO  (qtp1990662100-14486) [n:127.0.0.1:38726_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=db6777c4-94cc-4096-898c-3dd9b853a18c100752109001594&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1608682 INFO  (qtp207927362-14485) [n:127.0.0.1:42348_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=db6777c4-94cc-4096-898c-3dd9b853a18c&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 1609403 INFO  (OverseerThreadFactory-4867-thread-5-processing-n:127.0.0.1:42348_solr) [n:127.0.0.1:42348_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000009 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1609680 INFO  (OverseerCollectionConfigSetProcessor-72064196424105990-127.0.0.1:42348_solr-n_0000000000) [n:127.0.0.1:42348_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000009 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1609684 INFO  (qtp207927362-14482) [n:127.0.0.1:42348_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=db6777c4-94cc-4096-898c-3dd9b853a18c&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1609685 INFO  (qtp207927362-14482) [n:127.0.0.1:42348_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=db6777c4-94cc-4096-898c-3dd9b853a18c&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 1609685 INFO  (qtp207927362-14485) [n:127.0.0.1:42348_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :deletestatus with params requestid=db6777c4-94cc-4096-898c-3dd9b853a18c&action=DELETESTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1609686 INFO  (qtp207927362-14485) [n:127.0.0.1:42348_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=db6777c4-94cc-4096-898c-3dd9b853a18c&action=DELETESTATUS&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 1609687 INFO  (qtp207927362-14482) [n:127.0.0.1:42348_solr c:collection_two s:shard1 r:core_node2 x:collection_two_shard1_replica_n1] o.a.s.c.S.Request [collection_two_shard1_replica_n1]  webapp=/solr path=/select params={q=*:*&_stateVer_=collection_two:4&wt=javabin&version=2} hits=2 status=0 QTime=0
   [junit4]   2> 1609692 ERROR (TEST-AliasIntegrationTest.testDeleteOneOfTwoCollectionsAliased-seed#[B7B563BE2D2C5BDF]) [    ] o.a.s.c.s.i.CloudSolrClient Request to collection [collection_one] failed due to (404) org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:38726/solr/collection_one: Expected mime type application/octet-stream but got text/html. <html>
   [junit4]   2> <head>
   [junit4]   2> <meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
   [junit4]   2> <title>Error 404 Can not find: /solr/collection_one/select</title>
   [junit4]   2> </head>
   [junit4]   2> <body><h2>HTTP ERROR 404</h2>
   [junit4]   2> <p>Problem accessing /solr/collection_one/select. Reason:
   [junit4]   2> <pre>    Can not find: /solr/collection_one/select</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.8.v20171121</a><hr/>
   [junit4]   2> 
   [junit4]   2> </body>
   [junit4]   2> </html>
   [junit4]   2> , retry? 0
   [junit4]   2> 1609696 INFO  (qtp207927362-14482) [n:127.0.0.1:42348_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :deletealias with params async=6ef02ada-bcc1-4073-8c3c-3f524534729a&name=collection_alias_pair&action=DELETEALIAS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1609700 INFO  (qtp207927362-14482) [n:127.0.0.1:42348_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=6ef02ada-bcc1-4073-8c3c-3f524534729a&name=collection_alias_pair&action=DELETEALIAS&wt=javabin&version=2} status=0 QTime=4
   [junit4]   2> 1609700 INFO  (qtp207927362-14485) [n:127.0.0.1:42348_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=6ef02ada-bcc1-4073-8c3c-3f524534729a&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1609703 INFO  (qtp207927362-14485) [n:127.0.0.1:42348_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=6ef02ada-bcc1-4073-8c3c-3f524534729a&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=3
   [junit4]   2> 1609709 INFO  (OverseerThreadFactory-4867-thread-5-processing-n:127.0.0.1:42348_solr) [n:127.0.0.1:42348_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000010 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1610705 INFO  (qtp207927362-14482) [n:127.0.0.1:42348_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=6ef02ada-bcc1-4073-8c3c-3f524534729a&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1610706 INFO  (qtp207927362-14482) [n:127.0.0.1:42348_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=6ef02ada-bcc1-4073-8c3c-3f524534729a&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 1610707 INFO  (qtp207927362-14485) [n:127.0.0.1:42348_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :deletestatus with params requestid=6ef02ada-bcc1-4073-8c3c-3f524534729a&action=DELETESTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1610708 INFO  (qtp207927362-14485) [n:127.0.0.1:42348_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=6ef02ada-bcc1-4073-8c3c-3f524534729a&action=DELETESTATUS&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 1610708 INFO  (qtp207927362-14482) [n:127.0.0.1:42348_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params async=13b106b4-9c99-4619-8a6c-f5069774959c&name=collection_two&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1610712 INFO  (qtp207927362-14482) [n:127.0.0.1:42348_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=13b106b4-9c99-4619-8a6c-f5069774959c&name=collection_two&action=DELETE&wt=javabin&version=2} status=0 QTime=3
   [junit4]   2> 1610712 INFO  (qtp207927362-14485) [n:127.0.0.1:42348_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=13b106b4-9c99-4619-8a6c-f5069774959c&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1610715 INFO  (qtp207927362-14485) [n:127.0.0.1:42348_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=13b106b4-9c99-4619-8a6c-f5069774959c&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2
   [junit4]   2> 1610715 INFO  (OverseerThreadFactory-4867-thread-5-processing-n:127.0.0.1:42348_solr) [n:127.0.0.1:42348_solr    ] o.a.s.c.a.c.OverseerCollectionMessageHandler Executing Collection Cmd=action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true, asyncId=13b106b4-9c99-4619-8a6c-f5069774959c
   [junit4]   2> 1610716 INFO  (OverseerCollectionConfigSetProcessor-72064196424105990-127.0.0.1:42348_solr-n_0000000000) [n:127.0.0.1:42348_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000010 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1610716 INFO  (qtp207927362-14482) [n:127.0.0.1:42348_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=collection_two_shard1_replica_n1&async=13b106b4-9c99-4619-8a6c-f5069774959c100755147604458&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1610716 INFO  (parallelCoreAdminExecutor-4868-thread-2-processing-n:127.0.0.1:42348_solr 13b106b4-9c99-4619-8a6c-f5069774959c100755147604458 UNLOAD) [n:127.0.0.1:42348_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection_two.shard1.replica_n1, tag=null
   [junit4]   2> 1610716 INFO  (qtp207927362-14485) [n:127.0.0.1:42348_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : 13b106b4-9c99-4619-8a6c-f5069774959c100755147604458
   [junit4]   2> 1610716 INFO  (qtp207927362-14485) [n:127.0.0.1:42348_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=13b106b4-9c99-4619-8a6c-f5069774959c100755147604458&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1610716 INFO  (parallelCoreAdminExecutor-4868-thread-2-processing-n:127.0.0.1:42348_solr 13b106b4-9c99-4619-8a6c-f5069774959c100755147604458 UNLOAD) [n:127.0.0.1:42348_solr    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@74e9fe2d: rootName = solr_42348, domain = solr.core.collection_two.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.collection_two.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@30716408
   [junit4]   2> 1610723 INFO  (parallelCoreAdminExecutor-4868-thread-2-processing-n:127.0.0.1:42348_solr 13b106b4-9c99-4619-8a6c-f5069774959c100755147604458 UNLOAD) [n:127.0.0.1:42348_solr    ] o.a.s.c.SolrCore [collection_two_shard1_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@40dccdcb
   [junit4]   2> 1610723 INFO  (parallelCoreAdminExecutor-4868-thread-2-processing-n:127.0.0.1:42348_solr 13b106b4-9c99-4619-8a6c-f5069774959c100755147604458 UNLOAD) [n:127.0.0.1:42348_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection_two.shard1.replica_n1, tag=1088212427
   [junit4]   2> 1610723 INFO  (parallelCoreAdminExecutor-4868-thread-2-processing-n:127.0.0.1:42348_solr 13b106b4-9c99-4619-8a6c-f5069774959c100755147604458 UNLOAD) [n:127.0.0.1:42348_solr    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection_two.shard1.leader, tag=1088212427
   [junit4]   2> 1611719 INFO  (qtp207927362-14482) [n:127.0.0.1:42348_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=13b106b4-9c99-4619-8a6c-f5069774959c&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1611720 INFO  (qtp207927362-14485) [n:127.0.0.1:42348_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : 13b106b4-9c99-4619-8a6c-f5069774959c100755147604458
   [junit4]   2> 1611720 INFO  (qtp207927362-14485) [n:127.0.0.1:42348_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=13b106b4-9c99-4619-8a6c-f5069774959c100755147604458&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 1611721 INFO  (qtp207927362-14482) [n:127.0.0.1:42348_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=13b106b4-9c99-4619-8a6c-f5069774959c&action=REQUEST

[...truncated too long message...]

 from ZooKeeper... (1) -> (0)
   [junit4]   2> 1658519 INFO  (jetty-closer-4135-thread-1) [    ] o.a.s.c.Overseer Overseer (id=72064196424105990-127.0.0.1:42348_solr-n_0000000000) closing
   [junit4]   2> 1658519 INFO  (zkCallback-4152-thread-1-processing-n:127.0.0.1:42348_solr) [n:127.0.0.1:42348_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 1658519 INFO  (OverseerStateUpdate-72064196424105990-127.0.0.1:42348_solr-n_0000000000) [n:127.0.0.1:42348_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:42348_solr
   [junit4]   2> 1658523 WARN  (OverseerAutoScalingTriggerThread-72064196424105990-127.0.0.1:42348_solr-n_0000000000) [n:127.0.0.1:42348_solr    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 1658525 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10006013ce20006, likely client has closed socket
   [junit4]   2> 1658525 INFO  (jetty-closer-4135-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@242e6368{/solr,null,UNAVAILABLE}
   [junit4]   2> 1658528 INFO  (jetty-closer-4135-thread-1) [    ] o.e.j.s.session Stopped scavenging
   [junit4]   2> 1658529 INFO  (jetty-closer-4135-thread-2) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@33c17b1e{/solr,null,UNAVAILABLE}
   [junit4]   2> 1658531 INFO  (jetty-closer-4135-thread-2) [    ] o.e.j.s.session Stopped scavenging
   [junit4]   2> 1658532 ERROR (SUITE-AliasIntegrationTest-seed#[B7B563BE2D2C5BDF]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1658532 INFO  (SUITE-AliasIntegrationTest-seed#[B7B563BE2D2C5BDF]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:33924 33924
   [junit4]   2> 1668676 INFO  (Thread-3136) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:33924 33924
   [junit4]   2> 1668677 WARN  (Thread-3136) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	17	/solr/aliases.json
   [junit4]   2> 	17	/solr/clusterprops.json
   [junit4]   2> 	14	/solr/configs/conf
   [junit4]   2> 	7	/solr/collections/collection2meta/terms/shard1
   [junit4]   2> 	6	/solr/collections/collection1meta/terms/shard2
   [junit4]   2> 	6	/solr/collections/collection1meta/terms/shard1
   [junit4]   2> 	3	/solr/collections/collection_new/terms/shard1
   [junit4]   2> 	3	/solr/collections/collection_old/terms/shard2
   [junit4]   2> 	3	/solr/collections/testErrorChecks-collection/terms/shard1
   [junit4]   2> 	3	/solr/collections/collection1/terms/shard1
   [junit4]   2> 	2	/solr/collections/collection2/terms/shard1
   [junit4]   2> 	2	/solr/collections/collection_two/terms/shard1
   [junit4]   2> 	2	/solr/collections/collection_one/terms/shard1
   [junit4]   2> 	2	/solr/collections/collection_one/terms/shard2
   [junit4]   2> 	2	/solr/collections/collection_old/terms/shard1
   [junit4]   2> 	2	/solr/collections/testErrorChecks-collection/terms/shard2
   [junit4]   2> 	2	/solr/security.json
   [junit4]   2> 	2	/solr/collections/collection1/terms/shard2
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	185	/solr/aliases.json
   [junit4]   2> 	33	/solr/collections/collection1meta/state.json
   [junit4]   2> 	18	/solr/collections/collection2meta/state.json
   [junit4]   2> 	17	/solr/clusterstate.json
   [junit4]   2> 	11	/solr/collections/collection_one/state.json
   [junit4]   2> 	11	/solr/collections/collection1/state.json
   [junit4]   2> 	11	/solr/collections/collection_old/state.json
   [junit4]   2> 	11	/solr/collections/testErrorChecks-collection/state.json
   [junit4]   2> 	6	/solr/collections/collection_two/state.json
   [junit4]   2> 	6	/solr/collections/collection_new/state.json
   [junit4]   2> 	6	/solr/collections/collection2/state.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	152	/solr/collections
   [junit4]   2> 	17	/solr/live_nodes
   [junit4]   2> 
   [junit4]   2> NOTE: leaving temporary files on disk at: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.cloud.AliasIntegrationTest_B7B563BE2D2C5BDF-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {a_t=PostingsFormat(name=LuceneFixedGap), id=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128)))}, docValues:{}, maxPointsInLeafNode=1097, maxMBSortInHeap=7.745184829014363, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@57e2217c), locale=zh-SG, timezone=Australia/NSW
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_162 (64-bit)/cpus=3,threads=1,free=213044136,total=536870912
   [junit4]   2> NOTE: All tests run in this JVM: [LargeFieldTest, HttpSolrCallGetCoreTest, CdcrReplicationDistributedZkTest, TriLevelCompositeIdRoutingTest, PeerSyncTest, SpellCheckCollatorTest, CoreAdminCreateDiscoverTest, TestValueSourceCache, TestLMJelinekMercerSimilarityFactory, TestCustomDocTransformer, TestRandomFlRTGCloud, TestStressCloudBlindAtomicUpdates, TestDynamicFieldResource, TestIndexingPerformance, CleanupOldIndexTest, PrimitiveFieldTypeTest, TestDFRSimilarityFactory, PrimUtilsTest, TestAnalyzeInfixSuggestions, OverseerCollectionConfigSetProcessorTest, DeleteReplicaTest, AutoScalingHandlerTest, ActionThrottleTest, TestGeoJSONResponseWriter, TestClusterStateProvider, StressHdfsTest, TestStressVersions, SimpleFacetsTest, TestCoreAdminApis, TestSolrIndexConfig, TestNodeAddedTrigger, TestStressLucene, ReplaceNodeNoTargetTest, HdfsDirectoryFactoryTest, ChaosMonkeyNothingIsSafeTest, DistributedQueryComponentCustomSortTest, SuggestComponentContextFilterQueryTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, TestSchemaVersionResource, HdfsBasicDistributedZkTest, TestConfigSetsAPI, RequestHandlersTest, TestAnalyzedSuggestions, TestDynamicLoading, TestReqParamsAPI, TestChildDocTransformer, ConfigureRecoveryStrategyTest, SolrCLIZkUtilsTest, DistribDocExpirationUpdateProcessorTest, TestExclusionRuleCollectionAccess, TestCloudSearcherWarming, TestMinMaxOnMultiValuedField, TestSolrConfigHandlerCloud, CreateRoutedAliasTest, TestUniqueKeyFieldResource, RankQueryTest, DisMaxRequestHandlerTest, TestHdfsCloudBackupRestore, TestUtilizeNode, DistributedDebugComponentTest, CdcrRequestHandlerTest, TestBulkSchemaConcurrent, SolrGraphiteReporterTest, TestLegacyNumericUtils, NoCacheHeaderTest, UpdateParamsTest, AtomicUpdatesTest, NodeAddedTriggerTest, TestSortByMinMaxFunction, TestExceedMaxTermLength, ZkFailoverTest, TestSort, TestCodecSupport, TestWriterPerf, ClassificationUpdateProcessorIntegrationTest, TestSearchPerf, TestPolicyCloud, TestScoreJoinQPNoScore, SyncSliceTest, HDFSCollectionsAPITest, TestGenericDistributedQueue, ExecutePlanActionTest, UpdateLogTest, TestReversedWildcardFilterFactory, NotRequiredUniqueKeyTest, DirectUpdateHandlerTest, NumberUtilsTest, TestSkipOverseerOperations, BadComponentTest, TestDocumentBuilder, SystemInfoHandlerTest, TestDeleteCollectionOnDownNodes, TestFieldCacheReopen, DistanceFunctionTest, HdfsRestartWhileUpdatingTest, TestUninvertingReader, TestSolrConfigHandler, SpellCheckComponentTest, ReturnFieldsTest, TestFuzzyAnalyzedSuggestions, BadIndexSchemaTest, URLClassifyProcessorTest, SolrMetricsIntegrationTest, IndexSchemaTest, TestCloudJSONFacetJoinDomain, TestDistributedGrouping, LeaderFailureAfterFreshStartTest, TestCoreDiscovery, HttpPartitionTest, FieldAnalysisRequestHandlerTest, UUIDFieldTest, DeleteStatusTest, TestHdfsBackupRestoreCore, SimplePostToolTest, TestLegacyNumericRangeQueryBuilder, DistributedQueueTest, RequestLoggingTest, TestSolrCoreParser, BigEndianAscendingWordDeserializerTest, TestInfoStreamLogging, TestCollectionAPIs, TestHashQParserPlugin, TestFieldResource, TestSolrCLIRunExample, TestRandomDVFaceting, CollectionsAPIDistributedZkTest, CSVRequestHandlerTest, TestXmlQParserPlugin, ShowFileRequestHandlerTest, RollingRestartTest, MetricsHandlerTest, TestSearcherReuse, TestWordDelimiterFilterFactory, TestNoOpRegenerator, TestStressRecovery, CollectionStateFormat2Test, LoggingHandlerTest, AliasIntegrationTest]
   [junit4] Completed [234/778 (1!)] on J1 in 68.70s, 8 tests, 2 errors <<< FAILURES!

[...truncated 49691 lines...]
[repro] Jenkins log URL: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/1707/consoleText

[repro] Revision: 0424d9c06ba52037024ce5f0f678b2aca8c34fb7

[repro] Ant options: "-Dargs=-XX:-UseCompressedOops -XX:+UseG1GC"
[repro] Repro line:  ant test  -Dtestcase=AliasIntegrationTest -Dtests.method=testMetadata -Dtests.seed=B7B563BE2D2C5BDF -Dtests.slow=true -Dtests.locale=zh-SG -Dtests.timezone=Australia/NSW -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1

[repro] Repro line:  ant test  -Dtestcase=AliasIntegrationTest -Dtests.method=testModifyMetadataCAR -Dtests.seed=B7B563BE2D2C5BDF -Dtests.slow=true -Dtests.locale=zh-SG -Dtests.timezone=Australia/NSW -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1

[repro] ant clean

[...truncated 6 lines...]
[repro] Test suites by module:
[repro]    solr/core
[repro]       AliasIntegrationTest
[repro] ant compile-test

[...truncated 3313 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 -Dtests.class="*.AliasIntegrationTest" -Dtests.showOutput=onerror "-Dargs=-XX:-UseCompressedOops -XX:+UseG1GC" -Dtests.seed=B7B563BE2D2C5BDF -Dtests.slow=true -Dtests.locale=zh-SG -Dtests.timezone=Australia/NSW -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1

[...truncated 103 lines...]
[repro] Failures:
[repro]   0/5 failed: org.apache.solr.cloud.AliasIntegrationTest
[repro] Exiting with code 0

[...truncated 70 lines...]