You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2016/06/09 17:44:01 UTC

[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 637 - Failure!

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/637/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.common.cloud.TestCollectionStateWatchers.testWaitForStateWatcherIsRetainedOnPredicateFailure

Error Message:
Did not see a fully active cluster after 30 seconds

Stack Trace:
java.lang.AssertionError: Did not see a fully active cluster after 30 seconds
	at __randomizedtesting.SeedInfo.seed([39C7ABC96F36FD3E:B1F1099AB799152C]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.common.cloud.TestCollectionStateWatchers.testWaitForStateWatcherIsRetainedOnPredicateFailure(TestCollectionStateWatchers.java:227)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 13076 lines...]
   [junit4] Suite: org.apache.solr.common.cloud.TestCollectionStateWatchers
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.common.cloud.TestCollectionStateWatchers_39C7ABC96F36FD3E-001/init-core-data-001
   [junit4]   2> 185686 INFO  (SUITE-TestCollectionStateWatchers-seed#[39C7ABC96F36FD3E]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 185687 INFO  (SUITE-TestCollectionStateWatchers-seed#[39C7ABC96F36FD3E]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 185687 INFO  (Thread-409) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 185687 INFO  (Thread-409) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 185787 INFO  (SUITE-TestCollectionStateWatchers-seed#[39C7ABC96F36FD3E]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:54279
   [junit4]   2> 185787 INFO  (SUITE-TestCollectionStateWatchers-seed#[39C7ABC96F36FD3E]-worker) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 185788 INFO  (SUITE-TestCollectionStateWatchers-seed#[39C7ABC96F36FD3E]-worker) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 185791 INFO  (zkCallback-744-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@29622ae6 name:ZooKeeperConnection Watcher:127.0.0.1:54279 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 185791 INFO  (SUITE-TestCollectionStateWatchers-seed#[39C7ABC96F36FD3E]-worker) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 185791 INFO  (SUITE-TestCollectionStateWatchers-seed#[39C7ABC96F36FD3E]-worker) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 185791 INFO  (SUITE-TestCollectionStateWatchers-seed#[39C7ABC96F36FD3E]-worker) [    ] o.a.s.c.c.SolrZkClient makePath: /solr/solr.xml
   [junit4]   2> 185797 INFO  (jetty-launcher-743-thread-2) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 185798 INFO  (jetty-launcher-743-thread-3) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 185798 INFO  (jetty-launcher-743-thread-4) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 185798 INFO  (jetty-launcher-743-thread-1) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 185801 INFO  (jetty-launcher-743-thread-4) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1c51d046{/solr,null,AVAILABLE}
   [junit4]   2> 185803 INFO  (jetty-launcher-743-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@37559ff4{/solr,null,AVAILABLE}
   [junit4]   2> 185805 INFO  (jetty-launcher-743-thread-3) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@259602d2{/solr,null,AVAILABLE}
   [junit4]   2> 185806 INFO  (jetty-launcher-743-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@21442874{/solr,null,AVAILABLE}
   [junit4]   2> 185809 INFO  (jetty-launcher-743-thread-4) [    ] o.e.j.s.ServerConnector Started ServerConnector@6771baf4{HTTP/1.1,[http/1.1]}{127.0.0.1:41746}
   [junit4]   2> 185809 INFO  (jetty-launcher-743-thread-4) [    ] o.e.j.s.Server Started @195487ms
   [junit4]   2> 185809 INFO  (jetty-launcher-743-thread-4) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=41746}
   [junit4]   2> 185810 INFO  (jetty-launcher-743-thread-4) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6d06d69c
   [junit4]   2> 185810 INFO  (jetty-launcher-743-thread-4) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.common.cloud.TestCollectionStateWatchers_39C7ABC96F36FD3E-001/tempDir-001/node4'
   [junit4]   2> 185810 INFO  (jetty-launcher-743-thread-4) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 185810 INFO  (jetty-launcher-743-thread-4) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 185811 INFO  (jetty-launcher-743-thread-3) [    ] o.e.j.s.ServerConnector Started ServerConnector@142229c5{HTTP/1.1,[http/1.1]}{127.0.0.1:58078}
   [junit4]   2> 185811 INFO  (jetty-launcher-743-thread-3) [    ] o.e.j.s.Server Started @195488ms
   [junit4]   2> 185811 INFO  (jetty-launcher-743-thread-3) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=58078}
   [junit4]   2> 185811 INFO  (jetty-launcher-743-thread-1) [    ] o.e.j.s.ServerConnector Started ServerConnector@350c7e0{HTTP/1.1,[http/1.1]}{127.0.0.1:47050}
   [junit4]   2> 185811 INFO  (jetty-launcher-743-thread-1) [    ] o.e.j.s.Server Started @195489ms
   [junit4]   2> 185811 INFO  (jetty-launcher-743-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=47050}
   [junit4]   2> 185812 INFO  (jetty-launcher-743-thread-3) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6d06d69c
   [junit4]   2> 185812 INFO  (jetty-launcher-743-thread-3) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.common.cloud.TestCollectionStateWatchers_39C7ABC96F36FD3E-001/tempDir-001/node3'
   [junit4]   2> 185812 INFO  (jetty-launcher-743-thread-1) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6d06d69c
   [junit4]   2> 185812 INFO  (jetty-launcher-743-thread-3) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 185812 INFO  (jetty-launcher-743-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.common.cloud.TestCollectionStateWatchers_39C7ABC96F36FD3E-001/tempDir-001/node1'
   [junit4]   2> 185812 INFO  (jetty-launcher-743-thread-3) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 185812 INFO  (jetty-launcher-743-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 185812 INFO  (jetty-launcher-743-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 185812 INFO  (jetty-launcher-743-thread-3) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 185812 INFO  (jetty-launcher-743-thread-1) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 185812 INFO  (jetty-launcher-743-thread-3) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 185813 INFO  (jetty-launcher-743-thread-2) [    ] o.e.j.s.ServerConnector Started ServerConnector@3b03b85c{HTTP/1.1,[http/1.1]}{127.0.0.1:45294}
   [junit4]   2> 185813 INFO  (jetty-launcher-743-thread-2) [    ] o.e.j.s.Server Started @195490ms
   [junit4]   2> 185813 INFO  (jetty-launcher-743-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=45294}
   [junit4]   2> 185813 INFO  (jetty-launcher-743-thread-2) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@6d06d69c
   [junit4]   2> 185813 INFO  (jetty-launcher-743-thread-2) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.common.cloud.TestCollectionStateWatchers_39C7ABC96F36FD3E-001/tempDir-001/node2'
   [junit4]   2> 185813 INFO  (jetty-launcher-743-thread-2) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 185813 INFO  (jetty-launcher-743-thread-2) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 185813 INFO  (jetty-launcher-743-thread-2) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 185814 INFO  (jetty-launcher-743-thread-1) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 185814 INFO  (jetty-launcher-743-thread-4) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 185814 INFO  (jetty-launcher-743-thread-4) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 185816 INFO  (jetty-launcher-743-thread-2) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 185817 INFO  (zkCallback-745-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7b2cb7a1 name:ZooKeeperConnection Watcher:127.0.0.1:54279/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 185818 INFO  (jetty-launcher-743-thread-3) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 185818 INFO  (jetty-launcher-743-thread-3) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 185818 INFO  (zkCallback-747-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1f2452be name:ZooKeeperConnection Watcher:127.0.0.1:54279/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 185818 INFO  (zkCallback-746-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@12a2c40e name:ZooKeeperConnection Watcher:127.0.0.1:54279/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 185818 INFO  (jetty-launcher-743-thread-2) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 185818 INFO  (jetty-launcher-743-thread-1) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 185818 INFO  (jetty-launcher-743-thread-2) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 185818 INFO  (jetty-launcher-743-thread-1) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 185819 INFO  (jetty-launcher-743-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 185825 INFO  (jetty-launcher-743-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 185825 INFO  (jetty-launcher-743-thread-3) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 185826 INFO  (jetty-launcher-743-thread-2) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.common.cloud.TestCollectionStateWatchers_39C7ABC96F36FD3E-001/tempDir-001/node2/.
   [junit4]   2> 185826 INFO  (jetty-launcher-743-thread-2) [    ] o.a.s.c.CoreContainer New CoreContainer 48125253
   [junit4]   2> 185826 INFO  (jetty-launcher-743-thread-2) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.common.cloud.TestCollectionStateWatchers_39C7ABC96F36FD3E-001/tempDir-001/node2]
   [junit4]   2> 185826 WARN  (jetty-launcher-743-thread-2) [    ] o.a.s.c.CoreContainer Couldn't add files from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.common.cloud.TestCollectionStateWatchers_39C7ABC96F36FD3E-001/tempDir-001/node2/lib to classpath: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.common.cloud.TestCollectionStateWatchers_39C7ABC96F36FD3E-001/tempDir-001/node2/lib
   [junit4]   2> 185826 INFO  (jetty-launcher-743-thread-2) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,
   [junit4]   2> 185828 INFO  (jetty-launcher-743-thread-2) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: 
   [junit4]   2> 185829 INFO  (zkCallback-748-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@573c612d name:ZooKeeperConnection Watcher:127.0.0.1:54279/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 185830 INFO  (jetty-launcher-743-thread-2) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 185830 INFO  (jetty-launcher-743-thread-2) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 185830 INFO  (jetty-launcher-743-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54279/solr
   [junit4]   2> 185830 INFO  (jetty-launcher-743-thread-2) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 185830 INFO  (jetty-launcher-743-thread-2) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 185831 INFO  (jetty-launcher-743-thread-2) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 185839 INFO  (jetty-launcher-743-thread-4) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 185840 INFO  (jetty-launcher-743-thread-4) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 185840 INFO  (jetty-launcher-743-thread-3) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.common.cloud.TestCollectionStateWatchers_39C7ABC96F36FD3E-001/tempDir-001/node3/.
   [junit4]   2> 185840 INFO  (jetty-launcher-743-thread-3) [    ] o.a.s.c.CoreContainer New CoreContainer 225505788
   [junit4]   2> 185840 INFO  (jetty-launcher-743-thread-3) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.common.cloud.TestCollectionStateWatchers_39C7ABC96F36FD3E-001/tempDir-001/node3]
   [junit4]   2> 185841 INFO  (zkCallback-751-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@75d00e0d name:ZooKeeperConnection Watcher:127.0.0.1:54279 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 185841 WARN  (jetty-launcher-743-thread-3) [    ] o.a.s.c.CoreContainer Couldn't add files from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.common.cloud.TestCollectionStateWatchers_39C7ABC96F36FD3E-001/tempDir-001/node3/lib to classpath: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.common.cloud.TestCollectionStateWatchers_39C7ABC96F36FD3E-001/tempDir-001/node3/lib
   [junit4]   2> 185841 INFO  (jetty-launcher-743-thread-2) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 185841 INFO  (jetty-launcher-743-thread-3) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,
   [junit4]   2> 185841 INFO  (jetty-launcher-743-thread-2) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 185841 INFO  (jetty-launcher-743-thread-4) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 185841 INFO  (jetty-launcher-743-thread-3) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: 
   [junit4]   2> 185841 INFO  (jetty-launcher-743-thread-1) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.common.cloud.TestCollectionStateWatchers_39C7ABC96F36FD3E-001/tempDir-001/node1/.
   [junit4]   2> 185841 INFO  (jetty-launcher-743-thread-1) [    ] o.a.s.c.CoreContainer New CoreContainer 2096091831
   [junit4]   2> 185841 INFO  (jetty-launcher-743-thread-1) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.common.cloud.TestCollectionStateWatchers_39C7ABC96F36FD3E-001/tempDir-001/node1]
   [junit4]   2> 185841 WARN  (jetty-launcher-743-thread-1) [    ] o.a.s.c.CoreContainer Couldn't add files from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.common.cloud.TestCollectionStateWatchers_39C7ABC96F36FD3E-001/tempDir-001/node1/lib to classpath: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.common.cloud.TestCollectionStateWatchers_39C7ABC96F36FD3E-001/tempDir-001/node1/lib
   [junit4]   2> 185841 INFO  (jetty-launcher-743-thread-1) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,
   [junit4]   2> 185841 INFO  (jetty-launcher-743-thread-3) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 185841 INFO  (jetty-launcher-743-thread-3) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 185841 INFO  (jetty-launcher-743-thread-3) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54279/solr
   [junit4]   2> 185841 INFO  (jetty-launcher-743-thread-3) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 185841 INFO  (jetty-launcher-743-thread-3) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 185846 INFO  (jetty-launcher-743-thread-1) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: 
   [junit4]   2> 185856 INFO  (jetty-launcher-743-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$68/267283162@50e687ab
   [junit4]   2> 185857 INFO  (jetty-launcher-743-thread-4) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.common.cloud.TestCollectionStateWatchers_39C7ABC96F36FD3E-001/tempDir-001/node4/.
   [junit4]   2> 185857 INFO  (jetty-launcher-743-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 185857 INFO  (jetty-launcher-743-thread-1) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 185857 INFO  (jetty-launcher-743-thread-1) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 185857 INFO  (jetty-launcher-743-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54279/solr
   [junit4]   2> 185857 INFO  (jetty-launcher-743-thread-1) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 185857 INFO  (jetty-launcher-743-thread-1) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 185858 INFO  (jetty-launcher-743-thread-1) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 185859 INFO  (zkCallback-757-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1c431d2e name:ZooKeeperConnection Watcher:127.0.0.1:54279/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 185847 INFO  (jetty-launcher-743-thread-3) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 185860 INFO  (jetty-launcher-743-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 185857 INFO  (jetty-launcher-743-thread-4) [    ] o.a.s.c.CoreContainer New CoreContainer 751747190
   [junit4]   2> 185860 INFO  (jetty-launcher-743-thread-4) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.common.cloud.TestCollectionStateWatchers_39C7ABC96F36FD3E-001/tempDir-001/node4]
   [junit4]   2> 185860 WARN  (jetty-launcher-743-thread-4) [    ] o.a.s.c.CoreContainer Couldn't add files from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.common.cloud.TestCollectionStateWatchers_39C7ABC96F36FD3E-001/tempDir-001/node4/lib to classpath: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.common.cloud.TestCollectionStateWatchers_39C7ABC96F36FD3E-001/tempDir-001/node4/lib
   [junit4]   2> 185861 INFO  (jetty-launcher-743-thread-4) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,
   [junit4]   2> 185861 INFO  (jetty-launcher-743-thread-4) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: 
   [junit4]   2> 185862 INFO  (jetty-launcher-743-thread-4) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 185862 INFO  (jetty-launcher-743-thread-4) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 185862 INFO  (jetty-launcher-743-thread-4) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54279/solr
   [junit4]   2> 185862 INFO  (jetty-launcher-743-thread-4) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 185862 INFO  (jetty-launcher-743-thread-4) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 185863 INFO  (jetty-launcher-743-thread-4) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 185867 INFO  (jetty-launcher-743-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 185870 INFO  (zkCallback-761-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3e5416e5 name:ZooKeeperConnection Watcher:127.0.0.1:54279 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 185870 INFO  (jetty-launcher-743-thread-4) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 185871 INFO  (zkCallback-754-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@e1e87be name:ZooKeeperConnection Watcher:127.0.0.1:54279 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 185871 INFO  (zkCallback-758-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@57afa5ce name:ZooKeeperConnection Watcher:127.0.0.1:54279 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 185872 INFO  (jetty-launcher-743-thread-1) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 185872 INFO  (jetty-launcher-743-thread-1) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 185871 INFO  (jetty-launcher-743-thread-4) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 185875 INFO  (jetty-launcher-743-thread-3) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 185876 INFO  (jetty-launcher-743-thread-3) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 185878 INFO  (jetty-launcher-743-thread-1) [n:127.0.0.1:47050_solr    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$68/267283162@2be00fc1
   [junit4]   2> 185878 INFO  (jetty-launcher-743-thread-1) [n:127.0.0.1:47050_solr    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 185878 INFO  (jetty-launcher-743-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 185879 INFO  (jetty-launcher-743-thread-4) [n:127.0.0.1:41746_solr    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$68/267283162@6ca5bafc
   [junit4]   2> 185879 INFO  (jetty-launcher-743-thread-4) [n:127.0.0.1:41746_solr    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 185881 INFO  (zkCallback-763-thread-1-processing-n:127.0.0.1:41746_solr) [n:127.0.0.1:41746_solr    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1426c64a name:ZooKeeperConnection Watcher:127.0.0.1:54279/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 185881 INFO  (jetty-launcher-743-thread-4) [n:127.0.0.1:41746_solr    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 185885 INFO  (jetty-launcher-743-thread-3) [n:127.0.0.1:58078_solr    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$68/267283162@4bf26ab3
   [junit4]   2> 185886 INFO  (jetty-launcher-743-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 185887 INFO  (jetty-launcher-743-thread-3) [n:127.0.0.1:58078_solr    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 185887 INFO  (jetty-launcher-743-thread-4) [n:127.0.0.1:41746_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 185891 INFO  (jetty-launcher-743-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 185892 INFO  (jetty-launcher-743-thread-4) [n:127.0.0.1:41746_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 185893 INFO  (zkCallback-764-thread-1-processing-n:127.0.0.1:58078_solr) [n:127.0.0.1:58078_solr    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@162f10fd name:ZooKeeperConnection Watcher:127.0.0.1:54279/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 185893 INFO  (jetty-launcher-743-thread-3) [n:127.0.0.1:58078_solr    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 185895 INFO  (jetty-launcher-743-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 185896 INFO  (jetty-launcher-743-thread-4) [n:127.0.0.1:41746_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 185898 INFO  (jetty-launcher-743-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 185899 INFO  (jetty-launcher-743-thread-4) [n:127.0.0.1:41746_solr    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 185900 INFO  (jetty-launcher-743-thread-4) [n:127.0.0.1:41746_solr    ] o.a.s.c.c.SolrZkClient makePath: /collections
   [junit4]   2> 185900 INFO  (jetty-launcher-743-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.SolrZkClient makePath: /collections
   [junit4]   2> 185902 INFO  (jetty-launcher-743-thread-4) [n:127.0.0.1:41746_solr    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 185903 INFO  (jetty-launcher-743-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 185904 INFO  (jetty-launcher-743-thread-4) [n:127.0.0.1:41746_solr    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 185905 INFO  (jetty-launcher-743-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 185909 INFO  (jetty-launcher-743-thread-3) [n:127.0.0.1:58078_solr    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 185910 INFO  (jetty-launcher-743-thread-4) [n:127.0.0.1:41746_solr    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 185912 INFO  (jetty-launcher-743-thread-4) [n:127.0.0.1:41746_solr    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 185913 INFO  (jetty-launcher-743-thread-3) [n:127.0.0.1:58078_solr    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 185914 INFO  (jetty-launcher-743-thread-3) [n:127.0.0.1:58078_solr    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 185915 INFO  (jetty-launcher-743-thread-3) [n:127.0.0.1:58078_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 185918 INFO  (jetty-launcher-743-thread-3) [n:127.0.0.1:58078_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 185920 INFO  (jetty-launcher-743-thread-3) [n:127.0.0.1:58078_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 185921 INFO  (jetty-launcher-743-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 185921 INFO  (jetty-launcher-743-thread-3) [n:127.0.0.1:58078_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 185926 INFO  (jetty-launcher-743-thread-3) [n:127.0.0.1:58078_solr    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96042622910857227-127.0.0.1:58078_solr-n_0000000000
   [junit4]   2> 185926 INFO  (jetty-launcher-743-thread-3) [n:127.0.0.1:58078_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:58078_solr
   [junit4]   2> 185926 INFO  (jetty-launcher-743-thread-3) [n:127.0.0.1:58078_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 185928 INFO  (jetty-launcher-743-thread-3) [n:127.0.0.1:58078_solr    ] o.a.s.c.Overseer Overseer (id=96042622910857227-127.0.0.1:58078_solr-n_0000000000) starting
   [junit4]   2> 185930 INFO  (jetty-launcher-743-thread-3) [n:127.0.0.1:58078_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 185939 INFO  (jetty-launcher-743-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 185940 INFO  (jetty-launcher-743-thread-3) [n:127.0.0.1:58078_solr    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 185940 INFO  (zkCallback-762-thread-1-processing-n:127.0.0.1:47050_solr) [n:127.0.0.1:47050_solr    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3a2c2718 name:ZooKeeperConnection Watcher:127.0.0.1:54279/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 185940 INFO  (jetty-launcher-743-thread-1) [n:127.0.0.1:47050_solr    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 185941 INFO  (jetty-launcher-743-thread-3) [n:127.0.0.1:58078_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58078_solr
   [junit4]   2> 185942 INFO  (jetty-launcher-743-thread-3) [n:127.0.0.1:58078_solr    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:58078_solr
   [junit4]   2> 185954 INFO  (jetty-launcher-743-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 185955 INFO  (zkCallback-757-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 185975 INFO  (jetty-launcher-743-thread-3) [n:127.0.0.1:58078_solr    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 185975 INFO  (jetty-launcher-743-thread-3) [n:127.0.0.1:58078_solr    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 185976 INFO  (jetty-launcher-743-thread-3) [n:127.0.0.1:58078_solr    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.common.cloud.TestCollectionStateWatchers_39C7ABC96F36FD3E-001/tempDir-001/node3/.
   [junit4]   2> 185977 INFO  (jetty-launcher-743-thread-3) [n:127.0.0.1:58078_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions
   [junit4]   2> 185983 INFO  (OverseerCollectionConfigSetProcessor-96042622910857227-127.0.0.1:58078_solr-n_0000000000) [n:127.0.0.1:58078_solr    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 185984 INFO  (zkCallback-764-thread-1-processing-n:127.0.0.1:58078_solr) [n:127.0.0.1:58078_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 185985 INFO  (zkCallback-764-thread-1-processing-n:127.0.0.1:58078_solr) [n:127.0.0.1:58078_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 185986 INFO  (OverseerStateUpdate-96042622910857227-127.0.0.1:58078_solr-n_0000000000) [n:127.0.0.1:58078_solr    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 185986 INFO  (jetty-launcher-743-thread-3) [n:127.0.0.1:58078_solr    ] o.a.s.s.SolrDispatchFilter user.dir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1
   [junit4]   2> 185986 INFO  (jetty-launcher-743-thread-3) [n:127.0.0.1:58078_solr    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 185988 INFO  (jetty-launcher-743-thread-4) [n:127.0.0.1:41746_solr    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 185989 INFO  (OverseerStateUpdate-96042622910857227-127.0.0.1:58078_solr-n_0000000000) [n:127.0.0.1:58078_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (1)
   [junit4]   2> 185989 INFO  (jetty-launcher-743-thread-4) [n:127.0.0.1:41746_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 185989 INFO  (zkCallback-757-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 185991 INFO  (jetty-launcher-743-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 185992 INFO  (jetty-launcher-743-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96042622910857223-127.0.0.1:45294_solr-n_0000000001
   [junit4]   2> 185995 INFO  (jetty-launcher-743-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/96042622910857227-127.0.0.1:58078_solr-n_0000000000 to know if I could be the leader
   [junit4]   2> 185995 INFO  (jetty-launcher-743-thread-1) [n:127.0.0.1:47050_solr    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 185996 INFO  (jetty-launcher-743-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.ZkController Publish node=127.0.0.1:45294_solr as DOWN
   [junit4]   2> 185997 INFO  (jetty-launcher-743-thread-1) [n:127.0.0.1:47050_solr    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 185999 INFO  (OverseerStateUpdate-96042622910857227-127.0.0.1:58078_solr-n_0000000000) [n:127.0.0.1:58078_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:45294_solr"} current state version: 0
   [junit4]   2> 185999 INFO  (OverseerStateUpdate-96042622910857227-127.0.0.1:58078_solr-n_0000000000) [n:127.0.0.1:58078_solr    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:45294_solr
   [junit4]   2> 186001 INFO  (jetty-launcher-743-thread-4) [n:127.0.0.1:41746_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 186001 INFO  (jetty-launcher-743-thread-1) [n:127.0.0.1:47050_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 186002 INFO  (jetty-launcher-743-thread-4) [n:127.0.0.1:41746_solr    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96042622910857226-127.0.0.1:41746_solr-n_0000000002
   [junit4]   2> 186005 INFO  (jetty-launcher-743-thread-4) [n:127.0.0.1:41746_solr    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/96042622910857223-127.0.0.1:45294_solr-n_0000000001 to know if I could be the leader
   [junit4]   2> 186006 INFO  (jetty-launcher-743-thread-4) [n:127.0.0.1:41746_solr    ] o.a.s.c.ZkController Publish node=127.0.0.1:41746_solr as DOWN
   [junit4]   2> 186011 INFO  (OverseerStateUpdate-96042622910857227-127.0.0.1:58078_solr-n_0000000000) [n:127.0.0.1:58078_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:41746_solr"} current state version: 0
   [junit4]   2> 186011 INFO  (OverseerStateUpdate-96042622910857227-127.0.0.1:58078_solr-n_0000000000) [n:127.0.0.1:58078_solr    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:41746_solr
   [junit4]   2> 186012 INFO  (jetty-launcher-743-thread-1) [n:127.0.0.1:47050_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 186013 INFO  (jetty-launcher-743-thread-1) [n:127.0.0.1:47050_solr    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96042622910857228-127.0.0.1:47050_solr-n_0000000003
   [junit4]   2> 186014 INFO  (jetty-launcher-743-thread-1) [n:127.0.0.1:47050_solr    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/96042622910857226-127.0.0.1:41746_solr-n_0000000002 to know if I could be the leader
   [junit4]   2> 186015 INFO  (jetty-launcher-743-thread-1) [n:127.0.0.1:47050_solr    ] o.a.s.c.ZkController Publish node=127.0.0.1:47050_solr as DOWN
   [junit4]   2> 186020 INFO  (OverseerStateUpdate-96042622910857227-127.0.0.1:58078_solr-n_0000000000) [n:127.0.0.1:58078_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:47050_solr"} current state version: 0
   [junit4]   2> 186020 INFO  (OverseerStateUpdate-96042622910857227-127.0.0.1:58078_solr-n_0000000000) [n:127.0.0.1:58078_solr    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:47050_solr
   [junit4]   2> 187004 INFO  (jetty-launcher-743-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45294_solr
   [junit4]   2> 187005 INFO  (jetty-launcher-743-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:45294_solr
   [junit4]   2> 187007 INFO  (zkCallback-757-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 187007 INFO  (zkCallback-764-thread-2-processing-n:127.0.0.1:58078_solr) [n:127.0.0.1:58078_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 187007 INFO  (zkCallback-762-thread-1-processing-n:127.0.0.1:47050_solr) [n:127.0.0.1:47050_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 187007 INFO  (zkCallback-763-thread-1-processing-n:127.0.0.1:41746_solr) [n:127.0.0.1:41746_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 187008 INFO  (zkCallback-764-thread-2-processing-n:127.0.0.1:58078_solr) [n:127.0.0.1:58078_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 187008 INFO  (zkCallback-757-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 187009 INFO  (zkCallback-762-thread-1-processing-n:127.0.0.1:47050_solr) [n:127.0.0.1:47050_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 187009 INFO  (zkCallback-763-thread-1-processing-n:127.0.0.1:41746_solr) [n:127.0.0.1:41746_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 187009 INFO  (jetty-launcher-743-thread-4) [n:127.0.0.1:41746_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41746_solr
   [junit4]   2> 187010 INFO  (jetty-launcher-743-thread-4) [n:127.0.0.1:41746_solr    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:41746_solr
   [junit4]   2> 187012 INFO  (zkCallback-763-thread-1-processing-n:127.0.0.1:41746_solr) [n:127.0.0.1:41746_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 187012 INFO  (zkCallback-763-thread-1-processing-n:127.0.0.1:41746_solr) [n:127.0.0.1:41746_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 187012 INFO  (zkCallback-764-thread-2-processing-n:127.0.0.1:58078_solr) [n:127.0.0.1:58078_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 187012 INFO  (zkCallback-762-thread-1-processing-n:127.0.0.1:47050_solr) [n:127.0.0.1:47050_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 187013 INFO  (zkCallback-764-thread-2-processing-n:127.0.0.1:58078_solr) [n:127.0.0.1:58078_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 187013 INFO  (zkCallback-762-thread-1-processing-n:127.0.0.1:47050_solr) [n:127.0.0.1:47050_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 187013 INFO  (zkCallback-757-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 187014 INFO  (zkCallback-757-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 187020 INFO  (jetty-launcher-743-thread-1) [n:127.0.0.1:47050_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47050_solr
   [junit4]   2> 187020 INFO  (jetty-launcher-743-thread-1) [n:127.0.0.1:47050_solr    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:47050_solr
   [junit4]   2> 187024 INFO  (zkCallback-764-thread-2-processing-n:127.0.0.1:58078_solr) [n:127.0.0.1:58078_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [3])
   [junit4]   2> 187024 INFO  (zkCallback-762-thread-1-processing-n:127.0.0.1:47050_solr) [n:127.0.0.1:47050_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [3])
   [junit4]   2> 187024 INFO  (zkCallback-763-thread-1-processing-n:127.0.0.1:41746_solr) [n:127.0.0.1:41746_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [3])
   [junit4]   2> 187024 INFO  (zkCallback-757-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [3])
   [junit4]   2> 187025 INFO  (zkCallback-762-thread-1-processing-n:127.0.0.1:47050_solr) [n:127.0.0.1:47050_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 187025 INFO  (zkCallback-757-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 187025 INFO  (zkCallback-763-thread-1-processing-n:127.0.0.1:41746_solr) [n:127.0.0.1:41746_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 187031 INFO  (zkCallback-764-thread-2-processing-n:127.0.0.1:58078_solr) [n:127.0.0.1:58078_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 187035 INFO  (jetty-launcher-743-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 187035 INFO  (jetty-launcher-743-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 187036 INFO  (jetty-launcher-743-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.common.cloud.TestCollectionStateWatchers_39C7ABC96F36FD3E-001/tempDir-001/node2/.
   [junit4]   2> 187036 INFO  (jetty-launcher-743-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions
   [junit4]   2> 187037 INFO  (jetty-launcher-743-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.s.SolrDispatchFilter user.dir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1
   [junit4]   2> 187037 INFO  (jetty-launcher-743-thread-2) [n:127.0.0.1:45294_solr    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 187042 INFO  (jetty-launcher-743-thread-4) [n:127.0.0.1:41746_solr    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 187042 INFO  (jetty-launcher-743-thread-4) [n:127.0.0.1:41746_solr    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 187042 INFO  (jetty-launcher-743-thread-4) [n:127.0.0.1:41746_solr    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.common.cloud.TestCollectionStateWatchers_39C7ABC96F36FD3E-001/tempDir-001/node4/.
   [junit4]   2> 187043 INFO  (jetty-launcher-743-thread-4) [n:127.0.0.1:41746_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions
   [junit4]   2> 187043 INFO  (jetty-launcher-743-thread-4) [n:127.0.0.1:41746_solr    ] o.a.s.s.SolrDispatchFilter user.dir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1
   [junit4]   2> 187044 INFO  (jetty-launcher-743-thread-4) [n:127.0.0.1:41746_solr    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 187046 INFO  (jetty-launcher-743-thread-1) [n:127.0.0.1:47050_solr    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 187046 INFO  (jetty-launcher-743-thread-1) [n:127.0.0.1:47050_solr    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 187046 INFO  (jetty-launcher-743-thread-1) [n:127.0.0.1:47050_solr    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.common.cloud.TestCollectionStateWatchers_39C7ABC96F36FD3E-001/tempDir-001/node1/.
   [junit4]   2> 187046 INFO  (jetty-launcher-743-thread-1) [n:127.0.0.1:47050_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions
   [junit4]   2> 187047 INFO  (jetty-launcher-743-thread-1) [n:127.0.0.1:47050_solr    ] o.a.s.s.SolrDispatchFilter user.dir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1
   [junit4]   2> 187048 INFO  (jetty-launcher-743-thread-1) [n:127.0.0.1:47050_solr    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 187048 INFO  (SUITE-TestCollectionStateWatchers-seed#[39C7ABC96F36FD3E]-worker) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 187048 INFO  (SUITE-TestCollectionStateWatchers-seed#[39C7ABC96F36FD3E]-worker) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 187050 INFO  (zkCallback-769-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@57334e74 name:ZooKeeperConnection Watcher:127.0.0.1:54279 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 187051 INFO  (SUITE-TestCollectionStateWatchers-seed#[39C7ABC96F36FD3E]-worker) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 187051 INFO  (SUITE-TestCollectionStateWatchers-seed#[39C7ABC96F36FD3E]-worker) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 187052 INFO  (SUITE-TestCollectionStateWatchers-seed#[39C7ABC96F36FD3E]-worker) [    ] o.a.s.c.s.i.CloudSolrClient Final constructed zkHost string: 127.0.0.1:54279/solr
   [junit4]   2> 187053 INFO  (SUITE-TestCollectionStateWatchers-seed#[39C7ABC96F36FD3E]-worker) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 187054 INFO  (SUITE-TestCollectionStateWatchers-seed#[39C7ABC96F36FD3E]-worker) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 187056 INFO  (zkCallback-772-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5ce1c0bd name:ZooKeeperConnection Watcher:127.0.0.1:54279/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 187056 INFO  (SUITE-TestCollectionStateWatchers-seed#[39C7ABC96F36FD3E]-worker) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 187056 INFO  (SUITE-TestCollectionStateWatchers-seed#[39C7ABC96F36FD3E]-worker) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 187056 INFO  (SUITE-TestCollectionStateWatchers-seed#[39C7ABC96F36FD3E]-worker) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 187057 INFO  (SUITE-TestCollectionStateWatchers-seed#[39C7ABC96F36FD3E]-worker) [    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 187058 INFO  (SUITE-TestCollectionStateWatchers-seed#[39C7ABC96F36FD3E]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 187064 INFO  (SUITE-TestCollectionStateWatchers-seed#[39C7ABC96F36FD3E]-worker) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/config/solrconfig.xml
   [junit4]   2> 187068 INFO  (SUITE-TestCollectionStateWatchers-seed#[39C7ABC96F36FD3E]-worker) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/config/solrconfig-sql.xml
   [junit4]   2> 187070 INFO  (SUITE-TestCollectionStateWatchers-seed#[39C7ABC96F36FD3E]-worker) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/config/schema-replication1.xml
   [junit4]   2> 187073 INFO  (SUITE-TestCollectionStateWatchers-seed#[39C7ABC96F36FD3E]-worker) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/config/solrconfig-managed-schema.xml
   [junit4]   2> 187076 INFO  (SUITE-TestCollectionStateWatchers-seed#[39C7ABC96F36FD3E]-worker) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/config/schema-sql.xml
   [junit4]   2> 187078 INFO  (SUITE-TestCollectionStateWatchers-seed#[39C7ABC96F36FD3E]-worker) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/config/solrconfig-slave1.xml
   [junit4]   2> 187081 INFO  (SUITE-TestCollectionStateWatchers-seed#[39C7ABC96F36FD3E]-worker) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/config/schema.xml
   [junit4]   2> 187120 INFO  (TEST-TestCollectionStateWatchers.testWaitForStateWatcherIsRetainedOnPredicateFailure-seed#[39C7ABC96F36FD3E]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testWaitForStateWatcherIsRetainedOnPredicateFailure
   [junit4]   2> 187121 INFO  (TEST-TestCollectionStateWatchers.testWaitForStateWatcherIsRetainedOnPredicateFailure-seed#[39C7ABC96F36FD3E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 187121 INFO  (TEST-TestCollectionStateWatchers.testWaitForStateWatcherIsRetainedOnPredicateFailure-seed#[39C7ABC96F36FD3E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 187123 INFO  (zkCallback-773-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7c8bb7d8 name:ZooKeeperConnection Watcher:127.0.0.1:54279 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 187123 INFO  (TEST-TestCollectionStateWatchers.testWaitForStateWatcherIsRetainedOnPredicateFailure-seed#[39C7ABC96F36FD3E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 187124 INFO  (TEST-TestCollectionStateWatchers.testWaitForStateWatcherIsRetainedOnPredicateFailure-seed#[39C7ABC96F36FD3E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 187130 INFO  (qtp898352896-2377) [n:127.0.0.1:45294_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params async=106e2649-9d4e-4be3-9daa-3a707d829d43&replicationFactor=1&collection.configName=config&name=falsepredicate&action=CREATE&numShards=4&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 187133 INFO  (qtp898352896-2377) [n:127.0.0.1:45294_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=106e2649-9d4e-4be3-9daa-3a707d829d43&replicationFactor=1&collection.configName=config&name=falsepredicate&action=CREATE&numShards=4&wt=javabin&version=2} status=0 QTime=3
   [junit4]   2> 187134 INFO  (OverseerCollectionConfigSetProcessor-96042622910857227-127.0.0.1:58078_solr-n_0000000000) [n:127.0.0.1:58078_solr    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running/mn-106e2649-9d4e-4be3-9daa-3a707d829d43
   [junit4]   2> 187137 INFO  (OverseerCollectionConfigSetProcessor-96042622910857227-127.0.0.1:58078_solr-n_0000000000) [n:127.0.0.1:58078_solr    ] o.a.s.c.OverseerTaskProcessor Overseer Collection Message Handler: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
   [junit4]   2>   "name":"falsepredicate",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "collection.configName":"config",
   [junit4]   2>   "numShards":"4",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "async":"106e2649-9d4e-4be3-9daa-3a707d829d43",
   [junit4]   2>   "operation":"create"}
   [junit4]   2> 187137 INFO  (OverseerThreadFactory-534-thread-1-processing-n:127.0.0.1:58078_solr) [n:127.0.0.1:58078_solr    ] o.a.s.c.OverseerCollectionMessageHandler OverseerCollectionMessageHandler.processMessage : create , {
   [junit4]   2>   "name":"falsepredicate",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "collection.configName":"config",
   [junit4]   2>   "numShards":"4",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "async":"106e2649-9d4e-4be3-9daa-3a707d829d43",
   [junit4]   2>   "operation":"create"}
   [junit4]   2> 187137 INFO  (OverseerThreadFactory-534-thread-1-processing-n:127.0.0.1:58078_solr) [n:127.0.0.1:58078_solr    ] o.a.s.c.OverseerCollectionMessageHandler Create collection falsepredicate
   [junit4]   2> 187138 INFO  (OverseerThreadFactory-534-thread-1-processing-n:127.0.0.1:58078_solr) [n:127.0.0.1:58078_solr    ] o.a.s.c.OverseerCollectionMessageHandler creating collections conf node /collections/falsepredicate 
   [junit4]   2> 187139 INFO  (OverseerThreadFactory-534-thread-1-processing-n:127.0.0.1:58078_solr) [n:127.0.0.1:58078_solr    ] o.a.s.c.c.SolrZkClient makePath: /collections/falsepredicate
   [junit4]   2> 187140 INFO  (zkCallback-772-thread-1) [    ] o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 187141 INFO  (zkCallback-762-thread-1-processing-n:127.0.0.1:47050_solr) [n:127.0.0.1:47050_solr    ] o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 187141 INFO  (zkCallback-763-thread-1-processing-n:127.0.0.1:41746_solr) [n:127.0.0.1:41746_solr    ] o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 187141 INFO  (zkCallback-757-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 187141 INFO  (zkCallback-764-thread-2-processing-n:127.0.0.1:58078_solr) [n:127.0.0.1:58078_solr    ] o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 187143 INFO  (qtp898352896-2373) [n:127.0.0.1:45294_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=106e2649-9d4e-4be3-9daa-3a707d829d43&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 187145 INFO  (qtp898352896-2373) [n:127.0.0.1:45294_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=106e2649-9d4e-4be3-9daa-3a707d829d43&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=3
   [junit4]   2> 187147 INFO  (OverseerStateUpdate-96042622910857227-127.0.0.1:58078_solr-n_0000000000) [n:127.0.0.1:58078_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "name":"falsepredicate",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "replicationFactor":"1",
   [junit4]   2>   "collection.configName":"config",
   [junit4]   2>   "numShards":"4",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "async":"106e2649-9d4e-4be3-9daa-3a707d829d43",
   [junit4]   2>   "operation":"create"} current state version: 0
   [junit4]   2> 187147 INFO  (OverseerStateUpdate-96042622910857227-127.0.0.1:58078_solr-n_0000000000) [n:127.0.0.1:58078_solr    ] o.a.s.c.o.ClusterStateMutator building a new cName: falsepredicate
   [junit4]   2> 187148 INFO  (OverseerStateUpdate-96042622910857227-127.0.0.1:58078_solr-n_0000000000) [n:127.0.0.1:58078_solr    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/falsepredicate/state.json
   [junit4]   2> 187246 INFO  (OverseerThreadFactory-534-thread-1-processing-n:127.0.0.1:58078_solr) [n:127.0.0.1:58078_solr    ] o.a.s.c.OverseerCollectionMessageHandler Creating SolrCores for new collection falsepredicate, shardNames [shard1, shard2, shard3, shard4] , replicationFactor : 1
   [junit4]   2> 187246 INFO  (OverseerThreadFactory-534-thread-1-processing-n:127.0.0.1:58078_solr) [n:127.0.0.1:58078_solr    ] o.a.s.c.OverseerCollectionMessageHandler Creating core falsepredicate_shard3_replica1 as part of shard shard3 of collection falsepredicate on 127.0.0.1:58078_solr
   [junit4]   2> 187247 INFO  (OverseerThreadFactory-534-thread-1-processing-n:127.0.0.1:58078_solr) [n:127.0.0.1:58078_solr    ] o.a.s.c.OverseerCollectionMessageHandler Creating core falsepredicate_shard1_replica1 as part of shard shard1 of collection falsepredicate on 127.0.0.1:47050_solr
   [junit4]   2> 187247 INFO  (OverseerThreadFactory-534-thread-1-processing-n:127.0.0.1:58078_solr) [n:127.0.0.1:58078_solr    ] o.a.s.c.OverseerCollectionMessageHandler Creating core falsepredicate_shard2_replica1 as part of shard shard2 of collection falsepredicate on 127.0.0.1:41746_solr
   [junit4]   2> 187247 INFO  (OverseerThreadFactory-534-thread-1-processing-n:127.0.0.1:58078_solr) [n:127.0.0.1:58078_solr    ] o.a.s.c.OverseerCollectionMessageHandler Creating core falsepredicate_shard4_replica1 as part of shard shard4 of collection falsepredicate on 127.0.0.1:45294_solr
   [junit4]   2> 187257 INFO  (qtp692834948-2391) [n:127.0.0.1:41746_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=106e2649-9d4e-4be3-9daa-3a707d829d432953870792943327&qt=/admin/cores&collection.configName=config&name=falsepredicate_shard2_replica1&action=CREATE&numShards=4&collection=falsepredicate&shard=shard2&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 187258 INFO  (qtp780146536-2382) [n:127.0.0.1:58078_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=106e2649-9d4e-4be3-9daa-3a707d829d432953870792420111&qt=/admin/cores&collection.configName=config&name=falsepredicate_shard3_replica1&action=CREATE&numShards=4&collection=falsepredicate&shard=shard3&wt=javabin&version=2} status=0 QTime=4
   [junit4]   2> 187259 INFO  (qtp898352896-2375) [n:127.0.0.1:45294_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=106e2649-9d4e-4be3-9daa-3a707d829d432953870793212476&qt=/admin/cores&collection.configName=config&name=falsepredicate_shard4_replica1&action=CREATE&numShards=4&collection=falsepredicate&shard=shard4&wt=javabin&version=2} status=0 QTime=2
   [junit4]   2> 187259 INFO  (qtp1422561693-2401) [n:127.0.0.1:47050_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=106e2649-9d4e-4be3-9daa-3a707d829d432953870792758093&qt=/admin/cores&collection.configName=config&name=falsepredicate_shard1_replica1&action=CREATE&numShards=4&collection=falsepredicate&shard=shard1&wt=javabin&version=2} status=0 QTime=5
   [junit4]   2> 187259 INFO  (parallelCoreAdminExecutor-535-thread-1-processing-n:127.0.0.1:45294_solr 106e2649-9d4e-4be3-9daa-3a707d829d432953870793212476 CREATE) [n:127.0.0.1:45294_solr    ] o.a.s.h.a.CoreAdminOperation core create command async=106e2649-9d4e-4be3-9daa-3a707d829d432953870793212476&qt=/admin/cores&collection.configName=config&name=falsepredicate_shard4_replica1&action=CREATE&numShards=4&collection=falsepredicate&shard=shard4&wt=javabin&version=2
   [junit4]   2> 187259 INFO  (parallelCoreAdminExecutor-532-thread-1-processing-n:127.0.0.1:58078_solr 106e2649-9d4e-4be3-9daa-3a707d829d432953870792420111 CREATE) [n:127.0.0.1:58078_solr    ] o.a.s.h.a.CoreAdminOperation core create command async=106e2649-9d4e-4be3-9daa-3a707d829d432953870792420111&qt=/admin/cores&collection.configName=config&name=falsepredicate_shard3_replica1&action=CREATE&numShards=4&collection=falsepredicate&shard=shard3&wt=javabin&version=2
   [junit4]   2> 187260 INFO  (parallelCoreAdminExecutor-532-thread-1-processing-n:127.0.0.1:58078_solr 106e2649-9d4e-4be3-9daa-3a707d829d432953870792420111 CREATE) [n:127.0.0.1:58078_solr    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=falsepredicate_shard3_replica1, collection.configName=config, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=falsepredicate, numShards=4, dataDir=data/, shard=shard3}
   [junit4]   2> 187260 INFO  (parallelCoreAdminExecutor-535-thread-1-processing-n:127.0.0.1:45294_solr 106e2649-9d4e-4be3-9daa-3a707d829d432953870793212476 CREATE) [n:127.0.0.1:45294_solr    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=falsepredicate_shard4_replica1, collection.configName=config, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=falsepredicate, numShards=4, dataDir=data/, shard=shard4}
   [junit4]   2> 187260 INFO  (parallelCoreAdminExecutor-539-thread-1-processing-n:127.0.0.1:47050_solr 106e2649-9d4e-4be3-9daa-3a707d829d432953870792758093 CREATE) [n:127.0.0.1:47050_solr    ] o.a.s.h.a.CoreAdminOperation core create command async=106e2649-9d4e-4be3-9daa-3a707d829d432953870792758093&qt=/admin/cores&collection.configName=config&name=falsepredicate_shard1_replica1&action=CREATE&numShards=4&collection=falsepredicate&shard=shard1&wt=javabin&version=2
   [junit4]   2> 187261 INFO  (parallelCoreAdminExecutor-539-thread-1-processing-n:127.0.0.1:47050_solr 106e2649-9d4e-4be3-9daa-3a707d829d432953870792758093 CREATE) [n:127.0.0.1:47050_solr    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=falsepredicate_shard1_replica1, collection.configName=config, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=falsepredicate, numShards=4, dataDir=data/, shard=shard1}
   [junit4]   2> 187261 INFO  (parallelCoreAdminExecutor-535-thread-1-processing-n:127.0.0.1:45294_solr 106e2649-9d4e-4be3-9daa-3a707d829d432953870793212476 CREATE) [n:127.0.0.1:45294_solr c:falsepredicate s:shard4  x:falsepredicate_shard4_replica1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 187261 INFO  (parallelCoreAdminExecutor-537-thread-1-processing-n:127.0.0.1:41746_solr 106e2649-9d4e-4be3-9daa-3a707d829d432953870792943327 CREATE) [n:127.0.0.1:41746_solr    ] o.a.s.h.a.CoreAdminOperation core create command async=106e2649-9d4e-4be3-9daa-3a707d829d432953870792943327&qt=/admin/cores&collection.configName=config&name=falsepredicate_shard2_replica1&action=CREATE&numShards=4&collection=falsepredicate&shard=shard2&wt=javabin&version=2
   [junit4]   2> 187262 INFO  (parallelCoreAdminExecutor-537-thread-1-processing-n:127.0.0.1:41746_solr 106e2649-9d4e-4be3-9daa-3a707d829d432953870792943327 CREATE) [n:127.0.0.1:41746_solr    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=falsepredicate_shard2_replica1, collection.configName=config, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=falsepredicate, numShards=4, dataDir=data/, shard=shard2}
   [junit4]   2> 187262 INFO  (parallelCoreAdminExecutor-539-thread-1-processing-n:127.0.0.1:47050_solr 106e2649-9d4e-4be3-9daa-3a707d829d432953870792758093 CREATE) [n:127.0.0.1:47050_solr c:falsepredicate s:shard1  x:falsepredicate_shard1_replica1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 187262 INFO  (parallelCoreAdminExecutor-535-thread-1-processing-n:127.0.0.1:45294_solr 106e2649-9d4e-4be3-9daa-3a707d829d432953870793212476 CREATE) [n:127.0.0.1:45294_solr c:falsepredicate s:shard4  x:falsepredicate_shard4_replica1] o.a.s.c.ZkController Registering watch for collection falsepredicate
   [junit4]   2> 187263 INFO  (parallelCoreAdminExecutor-532-thread-1-processing-n:127.0.0.1:58078_solr 106e2649-9d4e-4be3-9daa-3a707d829d432953870792420111 CREATE) [n:127.0.0.1:58078_solr c:falsepredicate s:shard3  x:falsepredicate_shard3_replica1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 187263 INFO  (parallelCoreAdminExecutor-535-thread-1-processing-n:127.0.0.1:45294_solr 106e2649-9d4e-4be3-9daa-3a707d829d432953870793212476 CREATE) [n:127.0.0.1:45294_solr c:falsepredicate s:shard4  x:falsepredicate_shard4_replica1] o.a.s.c.c.ZkStateReader Add data for [falsepredicate] ver [0]
   [junit4]   2> 187263 INFO  (parallelCoreAdminExecutor-537-thread-1-processing-n:127.0.0.1:41746_solr 106e2649-9d4e-4be3-9daa-3a707d829d432953870792943327 CREATE) [n:127.0.0.1:41746_solr c:falsepredicate s:shard2  x:falsepredicate_shard2_replica1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 187264 INFO  (parallelCoreAdminExecutor-535-thread-1-processing-n:127.0.0.1:45294_solr 106e2649-9d4e-4be3-9daa-3a707d829d432953870793212476 CREATE) [n:127.0.0.1:45294_solr c:falsepredicate s:shard4  x:falsepredicate_shard4_replica1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 187265 INFO  (parallelCoreAdminExecutor-539-thread-1-processing-n:127.0.0.1:47050_solr 106e2649-9d4e-4be3-9daa-3a707d829d432953870792758093 CREATE) [n:127.0.0.1:47050_solr c:falsepredicate s:shard1  x:falsepredicate_shard1_replica1] o.a.s.c.ZkController Registering watch for collection falsepredicate
   [junit4]   2> 187265 INFO  (parallelCoreAdminExecutor-537-thread-1-processing-n:127.0.0.1:41746_solr 106e2649-9d4e-4be3-9daa-3a707d829d432953870792943327 CREATE) [n:127.0.0.1:41746_solr c:falsepredicate s:shard2  x:falsepredicate_shard2_replica1] o.a.s.c.ZkController Registering watch for collection falsepredicate
   [junit4]   2> 187265 INFO  (parallelCoreAdminExecutor-539-thread-1-processing-n:127.0.0.1:47050_solr 106e2649-9d4e-4be3-9daa-3a707d829d432953870792758093 CREATE) [n:127.0.0.1:47050_solr c:falsepredicate s:shard1  x:falsepredicate_shard1_replica1] o.a.s.c.c.ZkStateReader Add data for [falsepredicate] ver [0]
   [junit4]   2> 187266 INFO  (parallelCoreAdminExecutor-539-thread-1-processing-n:127.0.0.1:47050_solr 106e2649-9d4e-4be3-9daa-3a707d829d432953870792758093 CREATE) [n:127.0.0.1:47050_solr c:falsepredicate s:shard1  x:falsepredicate_shard1_replica1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 187266 INFO  (OverseerStateUpdate-96042622910857227-127.0.0.1:58078_solr-n_0000000000) [n:127.0.0.1:58078_solr    ] o.a.s.c.Overseer processMessage: queueSize: 4, message = {
   [junit4]   2>   "core":"falsepredicate_shard4_replica1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:45294/solr",
   [junit4]   2>   "node_name":"127.0.0.1:45294_solr",
   [junit4]   2>   "numShards":"4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard4",
   [junit4]   2>   "collection":"falsepredicate",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 187266 INFO  (OverseerStateUpdate-96042622910857227-127.0.0.1:58078_solr-n_0000000000) [n:127.0.0.1:58078_solr    ] o.a.s.c.o.ReplicaMutator Update state numShards=4 message={
   [junit4]   2>   "core":"falsepredicate_shard4_replica1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:45294/solr",
   [junit4]   2>   "node_name":"127.0.0.1:45294_solr",
   [junit4]   2>   "numShards":"4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard4",
   [junit4]   2>   "collection":"falsepredicate",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 187267 INFO  (qtp780146536-2383) [n:127.0.0.1:58078_solr    ] o.a.s.h.a.CoreAdminOperation Checking request status for : 106e2649-9d4e-4be3-9daa-3a707d829d432953870792420111
   [junit4]   2> 187267 INFO  (parallelCoreAdminExecutor-532-thread-1-processing-n:127.0.0.1:58078_solr 106e2649-9d4e-4be3-9daa-3a707d829d432953870792420111 CREATE) [n:127.0.0.1:58078_solr c:falsepredicate s:shard3  x:falsepredicate_shard3_replica1] o.a.s.c.ZkController Registering watch for collection falsepredicate
   [junit4]   2> 187267 INFO  (parallelCoreAdminExecutor-537-thread-1-processing-n:127.0.0.1:41746_solr 106e2649-9d4e-4be3-9daa-3a707d829d432953870792943327 CREATE) [n:127.0.0.1:41746_solr c:falsepredicate s:shard2  x:falsepredicate_shard2_replica1] o.a.s.c.c.ZkStateReader Add data for [falsepredicate] ver [0]
   [junit4]   2> 187267 INFO  (qtp780146536-2383) [n:127.0.0.1:58078_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=106e2649-9d4e-4be3-9daa-3a707d829d432953870792420111&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 187267 INFO  (parallelCoreAdminExecutor-537-thread-1-processing-n:127.0.0.1:41746_solr 106e2649-9d4e-4be3-9daa-3a707d829d432953870792943327 CREATE) [n:127.0.0.1:41746_solr c:falsepredicate s:shard2  x:falsepredicate_shard2_replica1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 187269 INFO  (parallelCoreAdminExecutor-532-thread-1-processing-n:127.0.0.1:58078_solr 106e2649-9d4e-4be3-9daa-3a707d829d432953870792420111 CREATE) [n:127.0.0.1:58078_solr c:falsepredicate s:shard3  x:falsepredicate_shard3_replica1] o.a.s.c.c.ZkStateReader Add data for [falsepredicate] ver [0]
   [junit4]   2> 187270 INFO  (parallelCoreAdminExecutor-532-thread-1-processing-n:127.0.0.1:58078_solr 106e2649-9d4e-4be3-9daa-3a707d829d432953870792420111 CREATE) [n:127.0.0.1:58078_solr c:falsepredicate s:shard3  x:falsepredicate_shard3_replica1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 187270 INFO  (OverseerStateUpdate-96042622910857227-127.0.0.1:58078_solr-n_0000000000) [n:127.0.0.1:58078_solr    ] o.a.s.c.Overseer processMessage: queueSize: 4, message = {
   [junit4]   2>   "core":"falsepredicate_shard1_replica1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:47050/solr",
   [junit4]   2>   "node_name":"127.0.0.1:47050_solr",
   [junit4]   2>   "numShards":"4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"falsepredicate",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 187271 INFO  (OverseerStateUpdate-96042622910857227-127.0.0.1:58078_solr-n_0000000000) [n:127.0.0.1:58078_solr    ] o.a.s.c.o.ReplicaMutator Update state numShards=4 message={
   [junit4]   2>   "core":"falsepredicate_shard1_replica1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:47050/solr",
   [junit4]   2>   "node_name":"127.0.0.1:47050_solr",
   [junit4]   2>   "numShards":"4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"falsepredicate",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 187273 INFO  (OverseerStateUpdate-96042622910857227-127.0.0.1:58078_solr-n_0000000000) [n:127.0.0.1:58078_solr    ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
   [junit4]   2>   "core":"falsepredicate_shard2_replica1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:41746/solr",
   [junit4]   2>   "node_name":"127.0.0.1:41746_solr",
   [junit4]   2>   "numShards":"4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "collection":"falsepredicate",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 187273 INFO  (OverseerStateUpdate-96042622910857227-127.0.0.1:58078_solr-n_0000000000) [n:127.0.0.1:58078_solr    ] o.a.s.c.o.ReplicaMutator Update state numShards=4 message={
   [junit4]   2>   "core":"falsepredicate_shard2_replica1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:41746/solr",
   [junit4]   2>   "node_name":"127.0.0.1:41746_solr",
   [junit4]   2>   "numShards":"4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "collection":"falsepredicate",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 187276 INFO  (OverseerStateUpdate-96042622910857227-127.0.0.1:58078_solr-n_0000000000) [n:127.0.0.1:58078_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"falsepredicate_shard3_replica1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:58078/solr",
   [junit4]   2>   "node_name":"127.0.0.1:58078_solr",
   [junit4]   2>   "numShards":"4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard3",
   [junit4]   2>   "collection":"falsepredicate",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 187276 INFO  (OverseerStateUpdate-96042622910857227-127.0.0.1:58078_solr-n_0000000000) [n:127.0.0.1:58078_solr    ] o.a.s.c.o.ReplicaMutator Update state numShards=4 message={
   [junit4]   2>   "core":"falsepredicate_shard3_replica1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:58078/solr",
   [junit4]   2>   "node_name":"127.0.0.1:58078_solr",
   [junit4]   2>   "numShards":"4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard3",
   [junit4]   2>   "collection":"falsepredicate",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 187379 INFO  (OverseerStateUpdate-96042622910857227-127.0.0.1:58078_solr-n_0000000000) [n:127.0.0.1:58078_solr    ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/falsepredicate/state.json version: 0
   [junit4]   2> 187380 INFO  (zkCallback-757-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/falsepredicate/state.json] for collection [falsepredicate] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 187380 INFO  (zkCallback-762-thread-1-processing-n:127.0.0.1:47050_solr) [n:127.0.0.1:47050_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/falsepredicate/state.json] for collection [falsepredicate] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 187380 INFO  (zkCallback-764-thread-2-processing-n:127.0.0.1:58078_solr) [n:127.0.0.1:58078_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/falsepredicate/state.json] for collection [falsepredicate] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 187380 INFO  (zkCallback-763-thread-1-processing-n:127.0.0.1:41746_solr) [n:127.0.0.1:41746_solr    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/falsepredicate/state.json] for collection [falsepredicate] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 187381 INFO  (zkCallback-757-thread-1-processing-n:127.0.0.1:45294_solr) [n:127.0.0.1:45294_solr    ] o.a.s.c.c.ZkStateReader Updating data for [falsepredicate] from [0] to [1]
   [junit4]   2> 187381 INFO  (zkCallback-762-thread-1-processing-n:127.0.0.1:47050_solr) [n:127.0.0.1:47050_solr    ] o.a.s.c.c.ZkStateReader Updating data for [falsepredicate] from [0] to [1]
   [junit4]   2> 187381 INFO  (zkCallback-763-thread-1-processing-n:127.0.0.1:41746_solr) [n:127.0.0.1:41746_solr    ] o.a.s.c.c.ZkStateReader Updating data for [falsepredicate] from [0] to [1]
   [junit4]   2> 187381 INFO  (zkCallback-764-thread-2-processing-n:127.0.0.1:58078_solr) [n:127.0.0.1:58078_solr    ] o.a.s.c.c.ZkStateReader Updating data for [falsepredicate] from [0] to [1]
   [junit4]   2> 188157 INFO  (qtp898352896-2378) [n:127.0.0.1:45294_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=106e2649-9d4e-4be3-9daa-3a707d829d43&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 188159 INFO  (qtp898352896-2378) [n:127.0.0.1:45294_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=106e2649-9d4e-4be3-9daa-3a707d829d43&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2
   [junit4]   2> 188265 INFO  (parallelCoreAdminExecutor-535-thread-1-processing-n:127.0.0.1:45294_solr 106e2649-9d4e-4be3-9daa-3a707d829d432953870793212476 CREATE) [n:127.0.0.1:45294_solr c:falsepredicate s:shard4  x:falsepredicate_shard4_replica1] o.a.s.c.ZkController waiting to find shard id in clusterstate for falsepredicate_shard4_replica1
   [junit4]   2> 188265 INFO  (parallelCoreAdminExecutor-535-thread-1-processing-n:127.0.0.1:45294_solr 106e2649-9d4e-4be3-9daa-3a707d829d432953870793212476 CREATE) [n:127.0.0.1:45294_solr c:falsepredicate s:shard4  x:falsepredicate_shard4_replica1] o.a.s.c.ZkController Check for collection zkNode:falsepredicate
   [junit4]   2> 188265 INFO  (parallelCoreAdminExecutor-535-thread-1-processing-n:127.0.0.1:45294_solr 106e2649-9d4e-4be3-9daa-3a707d829d432953870793212476 CREATE) [n:127.0.0.1:45294_solr c:falsepredicate s:shard4  x:falsepredicate_shard4_replica1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 188265 INFO  (parallelCoreAdminExecutor-535-thread-1-processing-n:127.0.0.1:45294_solr 106e2649-9d4e-4be3-9daa-3a707d829d432953870793212476 CREATE) [n:127.0.0.1:45294_solr c:falsepredicate s:shard4  x:falsepredicate_shard4_replica1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/falsepredicate]
   [junit4]   2> 188266 INFO  (parallelCoreAdminExecutor-539-thread-1-processing-n:127.0.0.1:47050_solr 106e2649-9d4e-4be3-9daa-3a707d829d432953870792758093 CREATE) [n:127.0.0.1:47050_solr c:falsepredicate s:shard1  x:falsepredicate_shard1_replica1] o.a.s.c.ZkController waiting to find shard id in clusterstate for falsepredicate_shard1_replica1
   [junit4]   2> 188266 INFO  (parallelCoreAdminExecutor-539-thread-1-processing-n:127.0.0.1:47050_solr 106e2649-9d4e-4be3-9daa-3a707d829d432953870792758093 CREATE) [n:127.0.0.1:47050_solr c:falsepredicate s:shard1  x:falsepredicate_shard1_replica1] o.a.s.c.ZkController Check for collection zkNode:falsepredicate
   [junit4]   2> 188267 INFO  (parallelCoreAdminExecutor-535-thread-1-processing-n:127.0.0.1:45294_solr 106e2649-9d4e-4be3-9daa-3a707d829d432953870793212476 CREATE) [n:127.0.0.1:45294_solr c:falsepredicate s:shard4  x:falsepredicate_shard4_replica1] o.a.s.c.c.ZkStateReader path=[/collections/falsepredicate] [configName]=[config] specified config exists in ZooKeeper
   [junit4]   2> 188267 INFO  (parallelCoreAdminExecutor-535-thread-1-processing-n:127.0.0.1:45294_solr 106e2649-9d4e-4be3-9daa-3a707d829d432953870793212476 CREATE) [n:127.0.0.1:45294_solr c:falsepredicate s:shard4  x:falsepredicate_shard4_replica1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.common.cloud.TestCollectionStateWatchers_39C7ABC96F36FD3E-001/tempDir-001/node2/falsepredicate_shard4_replica1'
   [junit4]   2> 188267 INFO  (parallelCoreAdminExecutor-535-thread-1-processing-n:127.0.0.1:45294_solr 106e2649-9d4e-4be3-9daa-3a707d829d432953870793212476 CREATE) [n:127.0.0.1:45294_solr c:falsepredicate s:shard4  x:falsepredicate_shard4_replica1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 188267 INFO  (parallelCoreAdminExecutor-535-thread-1-processing-n:127.0.0.1:45294_solr 106e2649-9d4e-4be3-9daa-3a707d829d432953870793212476 CREATE) [n:127.0.0.1:45294_solr c:falsepredicate s:shard4  x:falsepredicate_shard4_replica1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 188267 INFO  (parallelCoreAdminExecutor-539-thread-1-processing-n:127.0.0.1:47050_solr 106e2649-9d4e-4be3-9daa-3a707d829d432953870792758093 CREATE) [n:127.0.0.1:47050_solr c:falsepredicate s:shard1  x:falsepredicate_shard1_replica1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 188267 INFO  (parallelCoreAdminExecutor-539-thread-1-processing-n:127.0.0.1:47050_solr 106e2649-9d4e-4be3-9daa-3a707d829d432953870792758093 CREATE) [n:127.0.0.1:47050_solr c:falsepredicate s:shard1  x:falsepredicate_shard1_replica1] o.a.s

[...truncated too long message...]

te_shard1_replica1/data [CachedDir<<refCount=0;path=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.common.cloud.TestCollectionStateWatchers_39C7ABC96F36FD3E-001/tempDir-001/node3/./waitforstate_shard1_replica1/data;done=false>>]
   [junit4]   2> 274777 INFO  (coreCloseExecutor-642-thread-2) [n:127.0.0.1:58078_solr c:waitforstate s:shard1 r:core_node1 x:waitforstate_shard1_replica1] o.a.s.c.CachingDirectoryFactory Closing directory, CoreContainer#isShutdown=true
   [junit4]   2> 274777 INFO  (coreCloseExecutor-642-thread-2) [n:127.0.0.1:58078_solr c:waitforstate s:shard1 r:core_node1 x:waitforstate_shard1_replica1] o.a.s.c.CachingDirectoryFactory Closing directory: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.common.cloud.TestCollectionStateWatchers_39C7ABC96F36FD3E-001/tempDir-001/node3/./waitforstate_shard1_replica1/data
   [junit4]   2> 274777 INFO  (coreCloseExecutor-642-thread-2) [n:127.0.0.1:58078_solr c:waitforstate s:shard1 r:core_node1 x:waitforstate_shard1_replica1] o.a.s.c.CachingDirectoryFactory looking to close /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.common.cloud.TestCollectionStateWatchers_39C7ABC96F36FD3E-001/tempDir-001/node3/./waitforstate_shard1_replica1/data/index [CachedDir<<refCount=0;path=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.common.cloud.TestCollectionStateWatchers_39C7ABC96F36FD3E-001/tempDir-001/node3/./waitforstate_shard1_replica1/data/index;done=false>>]
   [junit4]   2> 274777 INFO  (coreCloseExecutor-642-thread-2) [n:127.0.0.1:58078_solr c:waitforstate s:shard1 r:core_node1 x:waitforstate_shard1_replica1] o.a.s.c.CachingDirectoryFactory Closing directory, CoreContainer#isShutdown=true
   [junit4]   2> 274777 INFO  (coreCloseExecutor-642-thread-2) [n:127.0.0.1:58078_solr c:waitforstate s:shard1 r:core_node1 x:waitforstate_shard1_replica1] o.a.s.c.CachingDirectoryFactory Closing directory: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.common.cloud.TestCollectionStateWatchers_39C7ABC96F36FD3E-001/tempDir-001/node3/./waitforstate_shard1_replica1/data/index
   [junit4]   2> 274775 INFO  (zkCallback-764-thread-7-processing-n:127.0.0.1:58078_solr) [n:127.0.0.1:58078_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
   [junit4]   2> 274777 INFO  (zkCallback-762-thread-3-processing-n:127.0.0.1:47050_solr) [n:127.0.0.1:47050_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [3])
   [junit4]   2> 274781 INFO  (jetty-launcher-743-thread-5) [    ] o.a.s.c.Overseer Overseer (id=96042622910857227-127.0.0.1:58078_solr-n_0000000000) closing
   [junit4]   2> 274783 INFO  (OverseerStateUpdate-96042622910857227-127.0.0.1:58078_solr-n_0000000000) [n:127.0.0.1:58078_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:58078_solr
   [junit4]   2> 274784 INFO  (zkCallback-764-thread-7-processing-n:127.0.0.1:58078_solr) [n:127.0.0.1:58078_solr    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 276266 WARN  (zkCallback-794-thread-1-processing-n:127.0.0.1:58363_solr) [n:127.0.0.1:58363_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 276267 INFO  (jetty-launcher-743-thread-8) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@672b3a4{/solr,null,UNAVAILABLE}
   [junit4]   2> 276287 WARN  (zkCallback-764-thread-7-processing-n:127.0.0.1:58078_solr) [n:127.0.0.1:58078_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 276287 WARN  (zkCallback-762-thread-3-processing-n:127.0.0.1:47050_solr) [n:127.0.0.1:47050_solr    ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
   [junit4]   2> 276288 INFO  (jetty-launcher-743-thread-6) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@21442874{/solr,null,UNAVAILABLE}
   [junit4]   2> 276288 INFO  (jetty-launcher-743-thread-5) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@259602d2{/solr,null,UNAVAILABLE}
   [junit4]   2> 276295 INFO  (SUITE-TestCollectionStateWatchers-seed#[39C7ABC96F36FD3E]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:54279 54279
   [junit4]   2> 281403 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxnFactory Ignoring unexpected runtime exception
   [junit4]   2> java.nio.channels.CancelledKeyException
   [junit4]   2> 	at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:73)
   [junit4]   2> 	at sun.nio.ch.SelectionKeyImpl.readyOps(SelectionKeyImpl.java:87)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:187)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 281405 INFO  (Thread-409) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:54279 54279
   [junit4]   2> 281412 WARN  (Thread-409) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	8	/solr/aliases.json
   [junit4]   2> 	8	/solr/clusterprops.json
   [junit4]   2> 	7	/solr/security.json
   [junit4]   2> 	6	/solr/configs/config
   [junit4]   2> 	3	/solr/collections/stateformat1/state.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	12	/solr/collections/waitforstate/state.json
   [junit4]   2> 	8	/solr/collections/falsepredicate/state.json
   [junit4]   2> 	8	/solr/clusterstate.json
   [junit4]   2> 	7	/solr/collections/testcollection/state.json
   [junit4]   2> 	3	/solr/collections/stateformat1/state.json
   [junit4]   2> 	2	/solr/collections/currentstate/state.json
   [junit4]   2> 	2	/solr/collections/delayed/state.json
   [junit4]   2> 	2	/solr/collections/tobedeleted/state.json
   [junit4]   2> 	2	/solr/overseer_elect/election/96042622910857227-127.0.0.1:58078_solr-n_0000000000
   [junit4]   2> 	2	/solr/overseer_elect/election/96042622910857228-127.0.0.1:47050_solr-n_0000000003
   [junit4]   2> 	2	/solr/overseer_elect/election/96042622910857223-127.0.0.1:45294_solr-n_0000000001
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	69	/solr/overseer/collection-queue-work
   [junit4]   2> 	63	/solr/overseer/queue
   [junit4]   2> 	13	/solr/overseer/queue-work
   [junit4]   2> 	8	/solr/collections
   [junit4]   2> 	5	/solr/live_nodes
   [junit4]   2> 
   [junit4]   2> 281412 INFO  (SUITE-TestCollectionStateWatchers-seed#[39C7ABC96F36FD3E]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-solrj/test/J1/temp/solr.common.cloud.TestCollectionStateWatchers_39C7ABC96F36FD3E-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene62): {}, docValues:{}, maxPointsInLeafNode=1853, maxMBSortInHeap=7.858242117050782, sim=RandomSimilarity(queryNorm=true,coord=yes): {}, locale=es-UY, timezone=Australia/Broken_Hill
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_92 (64-bit)/cpus=3,threads=1,free=56326336,total=154324992
   [junit4]   2> NOTE: All tests run in this JVM: [SolrExampleBinaryTest, MergeIndexesEmbeddedTest, LargeVolumeJettyTest, LargeVolumeBinaryJettyTest, LargeVolumeEmbeddedTest, TestJsonRecordReader, HttpSolrClientBuilderTest, SolrZkClientTest, TestFastInputStream, SolrQueryTest, NoOpResponseParserTest, SolrExampleStreamingBinaryTest, BasicHttpSolrClientTest, DocumentAnalysisResponseTest, StreamExpressionTest, SolrExampleJettyTest, CommonParamsTest, TestSuggesterResponse, SolrPingTest, TestConfigSetAdminRequest, SolrExampleXMLTest, SolrExampleEmbeddedTest, TestSpellCheckResponse, TestEmbeddedSolrServer, TestNamedListCodec, SolrDocumentTest, FieldAnalysisResponseTest, NamedListTest, FacetFieldTest, TestLBHttpSolrClient, GraphExpressionTest, TestCollectionAdminRequest, TestCollectionStateWatchers]
   [junit4] Completed [83/83 (1!)] on J1 in 95.75s, 9 tests, 1 failure <<< FAILURES!

[...truncated 10 lines...]
BUILD FAILED
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/build.xml:740: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/build.xml:684: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/build.xml:59: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build.xml:246: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/common-build.xml:531: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/common-build.xml:1427: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/common-build.xml:984: There were test failures: 83 suites, 641 tests, 1 failure, 2 ignored [seed: 39C7ABC96F36FD3E]

Total time: 89 minutes 40 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 639 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/639/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.update.HardAutoCommitTest.testCommitWithin

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([C556AE78FEB1CE6E:7F84C1007D9F207B]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:780)
	at org.apache.solr.update.HardAutoCommitTest.testCommitWithin(HardAutoCommitTest.java:99)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
	xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
</response>

	request was:q=id:529&qt=standard&start=0&rows=20&version=2.2
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:773)
	... 40 more




Build Log:
[...truncated 11515 lines...]
   [junit4] Suite: org.apache.solr.update.HardAutoCommitTest
   [junit4]   2> Creating dataDir: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.update.HardAutoCommitTest_C556AE78FEB1CE6E-001/init-core-data-001
   [junit4]   2> 2187727 INFO  (SUITE-HardAutoCommitTest-seed#[C556AE78FEB1CE6E]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 2187728 INFO  (SUITE-HardAutoCommitTest-seed#[C556AE78FEB1CE6E]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore
   [junit4]   2> 2187728 INFO  (SUITE-HardAutoCommitTest-seed#[C556AE78FEB1CE6E]-worker) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1'
   [junit4]   2> 2187728 INFO  (SUITE-HardAutoCommitTest-seed#[C556AE78FEB1CE6E]-worker) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2187728 INFO  (SUITE-HardAutoCommitTest-seed#[C556AE78FEB1CE6E]-worker) [    ] o.a.s.c.SolrResourceLoader using system property solr.solr.home: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr
   [junit4]   2> 2187728 INFO  (SUITE-HardAutoCommitTest-seed#[C556AE78FEB1CE6E]-worker) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 2187728 INFO  (SUITE-HardAutoCommitTest-seed#[C556AE78FEB1CE6E]-worker) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 2187755 INFO  (SUITE-HardAutoCommitTest-seed#[C556AE78FEB1CE6E]-worker) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2187761 WARN  (SUITE-HardAutoCommitTest-seed#[C556AE78FEB1CE6E]-worker) [    ] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 2187762 INFO  (SUITE-HardAutoCommitTest-seed#[C556AE78FEB1CE6E]-worker) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2187778 INFO  (SUITE-HardAutoCommitTest-seed#[C556AE78FEB1CE6E]-worker) [    ] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2187782 INFO  (SUITE-HardAutoCommitTest-seed#[C556AE78FEB1CE6E]-worker) [    ] o.a.s.s.IndexSchema [null] Schema name=test
   [junit4]   2> 2187862 INFO  (SUITE-HardAutoCommitTest-seed#[C556AE78FEB1CE6E]-worker) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2187867 INFO  (SUITE-HardAutoCommitTest-seed#[C556AE78FEB1CE6E]-worker) [    ] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2187870 INFO  (SUITE-HardAutoCommitTest-seed#[C556AE78FEB1CE6E]-worker) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2187885 INFO  (SUITE-HardAutoCommitTest-seed#[C556AE78FEB1CE6E]-worker) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2187903 INFO  (SUITE-HardAutoCommitTest-seed#[C556AE78FEB1CE6E]-worker) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2187956 INFO  (SUITE-HardAutoCommitTest-seed#[C556AE78FEB1CE6E]-worker) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2187956 INFO  (SUITE-HardAutoCommitTest-seed#[C556AE78FEB1CE6E]-worker) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2187956 INFO  (SUITE-HardAutoCommitTest-seed#[C556AE78FEB1CE6E]-worker) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2187956 INFO  (SUITE-HardAutoCommitTest-seed#[C556AE78FEB1CE6E]-worker) [    ] o.a.s.c.SolrResourceLoader using system property solr.solr.home: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr
   [junit4]   2> 2187956 INFO  (SUITE-HardAutoCommitTest-seed#[C556AE78FEB1CE6E]-worker) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr'
   [junit4]   2> 2187956 INFO  (SUITE-HardAutoCommitTest-seed#[C556AE78FEB1CE6E]-worker) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2187957 INFO  (SUITE-HardAutoCommitTest-seed#[C556AE78FEB1CE6E]-worker) [    ] o.a.s.c.SolrResourceLoader using system property solr.solr.home: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr
   [junit4]   2> 2187957 INFO  (SUITE-HardAutoCommitTest-seed#[C556AE78FEB1CE6E]-worker) [    ] o.a.s.c.CoreContainer New CoreContainer 1328500633
   [junit4]   2> 2187957 INFO  (SUITE-HardAutoCommitTest-seed#[C556AE78FEB1CE6E]-worker) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr]
   [junit4]   2> 2187957 WARN  (SUITE-HardAutoCommitTest-seed#[C556AE78FEB1CE6E]-worker) [    ] o.a.s.c.CoreContainer Couldn't add files from /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/lib to classpath: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/lib
   [junit4]   2> 2187959 INFO  (SUITE-HardAutoCommitTest-seed#[C556AE78FEB1CE6E]-worker) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 600000,connTimeout : 60000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,
   [junit4]   2> 2188005 INFO  (SUITE-HardAutoCommitTest-seed#[C556AE78FEB1CE6E]-worker) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: 
   [junit4]   2> 2188008 INFO  (SUITE-HardAutoCommitTest-seed#[C556AE78FEB1CE6E]-worker) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2188008 INFO  (SUITE-HardAutoCommitTest-seed#[C556AE78FEB1CE6E]-worker) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2188008 INFO  (SUITE-HardAutoCommitTest-seed#[C556AE78FEB1CE6E]-worker) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2188008 INFO  (SUITE-HardAutoCommitTest-seed#[C556AE78FEB1CE6E]-worker) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2188010 INFO  (SUITE-HardAutoCommitTest-seed#[C556AE78FEB1CE6E]-worker) [    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=collection1, dataDir=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.update.HardAutoCommitTest_C556AE78FEB1CE6E-001/init-core-data-001, shard=shard1}
   [junit4]   2> 2188010 INFO  (coreLoadExecutor-6424-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1'
   [junit4]   2> 2188011 INFO  (coreLoadExecutor-6424-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2188011 INFO  (coreLoadExecutor-6424-thread-1) [    ] o.a.s.c.SolrResourceLoader using system property solr.solr.home: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr
   [junit4]   2> 2188011 INFO  (coreLoadExecutor-6424-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 2188011 INFO  (coreLoadExecutor-6424-thread-1) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 2188033 INFO  (coreLoadExecutor-6424-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2188048 WARN  (coreLoadExecutor-6424-thread-1) [    ] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 2188050 INFO  (coreLoadExecutor-6424-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2188065 INFO  (coreLoadExecutor-6424-thread-1) [    ] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2188069 INFO  (coreLoadExecutor-6424-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2188423 INFO  (coreLoadExecutor-6424-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2188494 INFO  (coreLoadExecutor-6424-thread-1) [    ] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2188584 INFO  (coreLoadExecutor-6424-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2188587 INFO  (coreLoadExecutor-6424-thread-1) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2188589 INFO  (coreLoadExecutor-6424-thread-1) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2188590 INFO  (coreLoadExecutor-6424-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2188590 INFO  (coreLoadExecutor-6424-thread-1) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2188592 INFO  (coreLoadExecutor-6424-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1'
   [junit4]   2> 2188592 INFO  (coreLoadExecutor-6424-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1
   [junit4]   2> 2188592 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2188592 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.update.HardAutoCommitTest_C556AE78FEB1CE6E-001/init-core-data-001/]
   [junit4]   2> 2188592 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3bf8db93
   [junit4]   2> 2188593 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.update.HardAutoCommitTest_C556AE78FEB1CE6E-001/init-core-data-001
   [junit4]   2> 2188593 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.update.HardAutoCommitTest_C556AE78FEB1CE6E-001/init-core-data-001/index/
   [junit4]   2> 2188594 WARN  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.update.HardAutoCommitTest_C556AE78FEB1CE6E-001/init-core-data-001/index' doesn't exist. Creating new index...
   [junit4]   2> 2188597 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.update.HardAutoCommitTest_C556AE78FEB1CE6E-001/init-core-data-001/index
   [junit4]   2> 2188598 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=464512645, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 2188598 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@4f9222f8 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1be086e1),segFN=segments_1,generation=1}
   [junit4]   2> 2188598 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 2188598 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.update.HardAutoCommitTest_C556AE78FEB1CE6E-001/init-core-data-001/
   [junit4]   2> 2188605 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2188605 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2188605 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 2188605 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 2188605 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2188606 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2188606 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 2188606 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 2188606 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2188606 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2188606 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2188606 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2188608 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2188608 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2188610 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2188643 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2188651 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 2188652 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,spellCheckCompRH,lazy,/update/json/docs,/admin/luke,/export,spellCheckWithWordbreak_Direct,tvrh,/get,/admin/properties,/update/json,mltrh,/admin/threads,/search-facet-def,/update/csv,/sql,/graph,/search-facet-invariants,/admin/segments,spellCheckCompRH1,/mlt,/admin/system,/replication,dismax,defaults,/config,/stream,/schema,/admin/plugins,/admin/logging,spellCheckWithWordbreak,mock,/admin/ping,/update,/admin/file,/terms,spellCheckCompRH_Direct,/debug/dump
   [junit4]   2> 2188652 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 2188652 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2188652 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2188665 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2188665 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2188665 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=245443125, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 2188666 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@4f9222f8 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1be086e1),segFN=segments_1,generation=1}
   [junit4]   2> 2188666 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 2188666 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1da62538[collection1] main]
   [junit4]   2> 2188666 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageIO=org.apache.solr.rest.ManagedResourceStorage$InMemoryStorageIO}
   [junit4]   2> 2188666 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 2188666 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 2188666 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 2188846 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 2189323 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2189324 INFO  (searcherExecutor-6425-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 2189324 INFO  (searcherExecutor-6425-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 2189324 INFO  (searcherExecutor-6425-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 2189324 INFO  (searcherExecutor-6425-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 2189324 INFO  (searcherExecutor-6425-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 2189325 INFO  (searcherExecutor-6425-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 2189325 INFO  (searcherExecutor-6425-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 2189326 INFO  (searcherExecutor-6425-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 2189326 INFO  (searcherExecutor-6425-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 2189326 INFO  (searcherExecutor-6425-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1da62538[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2189327 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 2189327 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 2189327 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 2189327 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1536777887305170944
   [junit4]   2> 2189328 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.u.UpdateLog Took 1.0ms to seed version buckets with highest version 1536777887305170944
   [junit4]   2> 2189329 INFO  (coreLoadExecutor-6424-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 2189330 INFO  (SUITE-HardAutoCommitTest-seed#[C556AE78FEB1CE6E]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore end
   [junit4]   2> 2189349 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testCommitWithin
   [junit4]   2> 2189354 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{deleteByQuery=*:* (-1536777887328239616)} 0 4
   [junit4]   2> 2189355 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1'
   [junit4]   2> 2189355 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2189355 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    ] o.a.s.c.SolrResourceLoader using system property solr.solr.home: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr
   [junit4]   2> 2189356 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 2189356 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    ] o.a.s.c.SolrResourceLoader Adding 'file:/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 2189380 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2189386 WARN  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    ] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 2189387 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
   [junit4]   2> 2189402 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    ] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2189444 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2189533 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2189538 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    ] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2189539 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2189542 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2189543 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    ] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2189545 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2189545 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    ] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2189547 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1'
   [junit4]   2> 2189547 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    ] o.a.s.c.CoreContainer Reloading SolrCore 'collection1' using configuration from instancedir /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1
   [junit4]   2> 2189547 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/core/src/test-files/solr/collection1], dataDir=[/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.update.HardAutoCommitTest_C556AE78FEB1CE6E-001/init-core-data-001/]
   [junit4]   2> 2189548 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3bf8db93
   [junit4]   2> 2189548 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.update.HardAutoCommitTest_C556AE78FEB1CE6E-001/init-core-data-001/index/
   [junit4]   2> 2189548 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.update.HardAutoCommitTest_C556AE78FEB1CE6E-001/init-core-data-001/
   [junit4]   2> 2189551 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2189551 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2189551 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 2189551 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 2189552 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2189552 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2189552 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 2189552 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 2189552 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2189552 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2189552 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2189552 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2189555 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2189561 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2189565 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2189566 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2189575 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 2189577 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,spellCheckCompRH,lazy,/update/json/docs,/admin/luke,/export,spellCheckWithWordbreak_Direct,tvrh,/get,/admin/properties,/update/json,mltrh,/admin/threads,/search-facet-def,/update/csv,/sql,/graph,/search-facet-invariants,/admin/segments,spellCheckCompRH1,/mlt,/admin/system,/replication,dismax,defaults,/config,/stream,/schema,/admin/plugins,/admin/logging,spellCheckWithWordbreak,mock,/admin/ping,/update,/admin/file,/terms,spellCheckCompRH_Direct,/debug/dump
   [junit4]   2> 2189577 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 2189577 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2189578 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2189578 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@10f03560[collection1] main]
   [junit4]   2> 2189578 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageIO=org.apache.solr.rest.ManagedResourceStorage$InMemoryStorageIO}
   [junit4]   2> 2189578 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using InMemoryStorage
   [junit4]   2> 2189578 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 2189578 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 2189579 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 2189588 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2189589 INFO  (searcherExecutor-6430-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 2189589 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 2189589 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 2189589 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 2189589 INFO  (searcherExecutor-6430-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 2189589 INFO  (searcherExecutor-6430-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 2189589 INFO  (searcherExecutor-6430-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 2189590 INFO  (searcherExecutor-6430-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 2189591 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.u.UpdateLog Took 1.0ms to seed version buckets with highest version 1536777887328239616
   [junit4]   2> 2189591 INFO  (searcherExecutor-6430-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 2189591 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.u.DefaultSolrCoreState Closing old IndexWriter... core=collection1
   [junit4]   2> 2189592 INFO  (searcherExecutor-6430-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 2189592 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=22, maxMergeAtOnceExplicit=38, maxMergedSegmentMB=42.19921875, floorSegmentMB=0.685546875, forceMergeDeletesPctAllowed=7.042521221606199, segmentsPerTier=46.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 2189592 INFO  (searcherExecutor-6430-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 2189593 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@4f9222f8 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1be086e1),segFN=segments_1,generation=1}
   [junit4]   2> 2189594 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 2189594 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
   [junit4]   2> 2189594 WARN  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.c.SolrCore [collection1] PERFORMANCE WARNING: Overlapping onDeckSearchers=2
   [junit4]   2> 2189594 INFO  (searcherExecutor-6430-thread-1-processing-x:collection1) [    x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 2189595 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@14d17c6f[collection1] main]
   [junit4]   2> 2189596 INFO  (searcherExecutor-6430-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@10f03560[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2189596 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.c.CoreContainer replacing core: collection1
   [junit4]   2> 2189596 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@739a96cd
   [junit4]   2> 2189596 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.u.DirectUpdateHandler2 closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=1,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=0}
   [junit4]   2> 2189597 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 2189604 INFO  (searcherExecutor-6430-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@14d17c6f[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2189605 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[529 (1536777887594577920)]} 0 1
   [junit4]   2> 2189606 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:529&qt=standard&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 2189606 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {add=[530 (1536777887597723648)]} 0 3
   [junit4]   2> 2189606 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:530&qt=standard&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 2189607 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] {delete=[529 (-1536777887598772224)]} 0 3
   [junit4]   2> 2189607 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=id:529&qt=standard&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 2189607 ERROR (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.SolrTestCaseJ4 REQUEST FAILED: xpath=//result[@numFound=1]
   [junit4]   2> 	xml response was: <?xml version="1.0" encoding="UTF-8"?>
   [junit4]   2> <response>
   [junit4]   2> <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
   [junit4]   2> </response>
   [junit4]   2> 
   [junit4]   2> 	request was:q=id:529&qt=standard&start=0&rows=20&version=2.2
   [junit4]   2> 2189607 ERROR (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.SolrTestCaseJ4 REQUEST FAILED: q=id:529&qt=standard&start=0&rows=20&version=2.2:java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
   [junit4]   2> 	xml response was: <?xml version="1.0" encoding="UTF-8"?>
   [junit4]   2> <response>
   [junit4]   2> <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
   [junit4]   2> </response>
   [junit4]   2> 
   [junit4]   2> 	request was:q=id:529&qt=standard&start=0&rows=20&version=2.2
   [junit4]   2> 	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:773)
   [junit4]   2> 	at org.apache.solr.update.HardAutoCommitTest.testCommitWithin(HardAutoCommitTest.java:99)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:498)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 2189608 INFO  (TEST-HardAutoCommitTest.testCommitWithin-seed#[C556AE78FEB1CE6E]) [    x:collection1] o.a.s.SolrTestCaseJ4 ###Ending testCommitWithin
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=HardAutoCommitTest -Dtests.method=testCommitWithin -Dtests.seed=C556AE78FEB1CE6E -Dtests.slow=true -Dtests.locale=pt-PT -Dtests.timezone=Canada/Central -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   0.36s J1 | HardAutoCommitTest.testCommitWithin <<<
   [junit4]    > Throwable #1: java.lang.RuntimeException: Exception during query
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([C556AE78FEB1CE6E:7F84C1007D9F207B]:0)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:780)
   [junit4]    > 	at org.apache.solr.update.HardAutoCommitTest.testCommitWithin(HardAutoCommitTest.java:99)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]    > Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
   [junit4]    > 	xml response was: <?xml version="1.0" encoding="UTF-8"?>
   [junit4]    > <response>
   [junit4]    > <lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
   [junit4]    > </response>
   [junit4]    > 	request was:q=id:529&qt=standard&start=0&rows=20&version=2.2
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:773)
   [junit4]    > 	... 40 more
   [junit4]   2> 2189705 INFO  (SUITE-HardAutoCommitTest-seed#[C556AE78FEB1CE6E]-worker) [    x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> 2189705 INFO  (SUITE-HardAutoCommitTest-seed#[C556AE78FEB1CE6E]-worker) [    x:collection1] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1328500633
   [junit4]   2> 2189705 INFO  (coreCloseExecutor-6434-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@de0000d
   [junit4]   2> 2189706 INFO  (coreCloseExecutor-6434-thread-1-processing-x:collection1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=2,adds=2,deletesById=1,deletesByQuery=0,errors=0,cumulative_adds=2,cumulative_deletesById=1,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=0}
   [junit4]   2> 2189706 INFO  (coreCloseExecutor-6434-thread-1-processing-x:collection1) [    x:collection1] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 2189706 INFO  (coreCloseExecutor-6434-thread-1-processing-x:collection1) [    x:collection1] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 2189706 INFO  (coreCloseExecutor-6434-thread-1-processing-x:collection1) [    x:collection1] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 2189706 INFO  (coreCloseExecutor-6434-thread-1-processing-x:collection1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter close.
   [junit4]   2> 2189710 INFO  (coreCloseExecutor-6434-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@4f9222f8 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1be086e1),segFN=segments_1,generation=1}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@4f9222f8 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1be086e1),segFN=segments_2,generation=2}
   [junit4]   2> 2189710 INFO  (coreCloseExecutor-6434-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 2
   [junit4]   2> 2189710 INFO  (coreCloseExecutor-6434-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 2189766 INFO  (coreCloseExecutor-6434-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 2189766 INFO  (coreCloseExecutor-6434-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.update.HardAutoCommitTest_C556AE78FEB1CE6E-001/init-core-data-001 [CachedDir<<refCount=0;path=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.update.HardAutoCommitTest_C556AE78FEB1CE6E-001/init-core-data-001;done=false>>]
   [junit4]   2> 2189766 INFO  (coreCloseExecutor-6434-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory, CoreContainer#isShutdown=true
   [junit4]   2> 2189766 INFO  (coreCloseExecutor-6434-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.update.HardAutoCommitTest_C556AE78FEB1CE6E-001/init-core-data-001
   [junit4]   2> 2189766 INFO  (coreCloseExecutor-6434-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.update.HardAutoCommitTest_C556AE78FEB1CE6E-001/init-core-data-001/index [CachedDir<<refCount=0;path=/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.update.HardAutoCommitTest_C556AE78FEB1CE6E-001/init-core-data-001/index;done=false>>]
   [junit4]   2> 2189767 INFO  (coreCloseExecutor-6434-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory, CoreContainer#isShutdown=true
   [junit4]   2> 2189767 INFO  (coreCloseExecutor-6434-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.update.HardAutoCommitTest_C556AE78FEB1CE6E-001/init-core-data-001/index
   [junit4]   2> NOTE: leaving temporary files on disk at: /export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build/solr-core/test/J1/temp/solr.update.HardAutoCommitTest_C556AE78FEB1CE6E-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene62), sim=RandomSimilarity(queryNorm=false,coord=yes): {}, locale=pt-PT, timezone=Canada/Central
   [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_92 (64-bit)/cpus=3,threads=1,free=76609776,total=499122176
   [junit4]   2> NOTE: All tests run in this JVM: [TestSystemIdResolver, SuggestComponentContextFilterQueryTest, TestBadConfig, TestStressRecovery, DefaultValueUpdateProcessorTest, TestTolerantUpdateProcessorCloud, TestIndexSearcher, CheckHdfsIndexTest, LeaderFailoverAfterPartitionTest, SignatureUpdateProcessorFactoryTest, DistributedQueueTest, SolrCloudExampleTest, SolrCmdDistributorTest, CollectionsAPIDistributedZkTest, TestCloudPivotFacet, SolrInfoMBeanTest, TestManagedStopFilterFactory, TestReloadAndDeleteDocs, DistributedFacetPivotWhiteBoxTest, AnalysisErrorHandlingTest, BasicZkTest, TestSuggestSpellingConverter, OverriddenZkACLAndCredentialsProvidersTest, TestFreeTextSuggestions, TestBinaryResponseWriter, TestTrie, BasicFunctionalityTest, FieldAnalysisRequestHandlerTest, TestQueryUtils, DateMathParserTest, TestSerializedLuceneMatchVersion, TestHashQParserPlugin, TestUtils, TestCollapseQParserPlugin, TestFastWriter, CoreAdminHandlerTest, TestAtomicUpdateErrorCases, PluginInfoTest, NotRequiredUniqueKeyTest, TestSolrJ, RemoteQueryErrorTest, TestBlendedInfixSuggestions, JavabinLoaderTest, SuggesterTest, BlockJoinFacetDistribTest, TestRTGBase, TestRecoveryHdfs, BigEndianAscendingWordSerializerTest, CleanupOldIndexTest, SortByFunctionTest, FileUtilsTest, TestCustomSort, TestTestInjection, UniqFieldsUpdateProcessorFactoryTest, TestJettySolrRunner, CdcrVersionReplicationTest, TestPKIAuthenticationPlugin, TestSolrQueryParserResource, TestSortByMinMaxFunction, TestScoreJoinQPNoScore, ResponseLogComponentTest, TestSolrCoreProperties, TestStressCloudBlindAtomicUpdates, TestUninvertingReader, TestWriterPerf, SpellCheckCollatorTest, TestPivotHelperCode, TestReplicationHandlerBackup, QueryParsingTest, SearchHandlerTest, TestFieldCache, TestSolrDeletionPolicy2, BasicDistributedZk2Test, TestSolrConfigHandlerCloud, HdfsLockFactoryTest, HdfsTlogReplayBufferedWhileIndexingTest, TestReload, HLLSerializationTest, SharedFSAutoReplicaFailoverUtilsTest, TestMiniSolrCloudCluster, DirectSolrConnectionTest, TestRawTransformer, DateFieldTest, TestManagedSchemaAPI, TestDistribDocBasedVersion, DeleteReplicaTest, DistribDocExpirationUpdateProcessorTest, VersionInfoTest, DistanceUnitsTest, TestFiltering, TestQueryWrapperFilter, CdcrReplicationHandlerTest, TestFieldTypeCollectionResource, DistributedFacetPivotSmallAdvancedTest, TestMinMaxOnMultiValuedField, TestValueSourceCache, SolrIndexSplitterTest, TestStressUserVersions, AsyncCallRequestStatusResponseTest, TestBM25SimilarityFactory, DistributedDebugComponentTest, BufferStoreTest, TestRecovery, TestDefaultStatsCache, DistanceFunctionTest, SolrCoreCheckLockOnStartupTest, SpellingQueryConverterTest, TestSolrXml, DistributedFacetPivotLargeTest, FieldMutatingUpdateProcessorTest, TestRandomFaceting, TestSha256AuthenticationProvider, TestFieldCollectionResource, MultiThreadedOCPTest, AnalyticsMergeStrategyTest, TestRestoreCore, TestDistributedStatsComponentCardinality, EchoParamsTest, TestCrossCoreJoin, TestFoldingMultitermQuery, TestPerFieldSimilarityWithDefaultOverride, OverseerRolesTest, TestFilteredDocIdSet, TestFuzzyAnalyzedSuggestions, BaseCdcrDistributedZkTest, DirectUpdateHandlerOptimizeTest, ExplicitHLLTest, ZkNodePropsTest, CopyFieldTest, TestManagedResourceStorage, AliasIntegrationTest, TestSizeLimitedDistributedMap, ConnectionReuseTest, CursorMarkTest, SecurityConfHandlerTest, TlogReplayBufferedWhileIndexingTest, TestRangeQuery, OpenCloseCoreStressTest, TolerantUpdateProcessorTest, PKIAuthenticationIntegrationTest, OpenExchangeRatesOrgProviderTest, AtomicUpdatesTest, TestUseDocValuesAsStored, TestRequestForwarding, EnumFieldTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, SolrPluginUtilsTest, SpellPossibilityIteratorTest, TestReversedWildcardFilterFactory, TestPerFieldSimilarity, BasicDistributedZkTest, ChaosMonkeySafeLeaderTest, FullSolrCloudDistribCmdsTest, ClusterStateUpdateTest, ZkSolrClientTest, ShardRoutingCustomTest, AutoCommitTest, TestDistributedGrouping, TermVectorComponentDistributedTest, TestJoin, HardAutoCommitTest]
   [junit4] Completed [327/613 (1!)] on J1 in 2.06s, 1 test, 1 error <<< FAILURES!

[...truncated 895 lines...]
BUILD FAILED
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/build.xml:740: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/build.xml:684: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/build.xml:59: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/build.xml:233: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/solr/common-build.xml:531: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/common-build.xml:1427: The following error occurred while executing this line:
/export/home/jenkins/workspace/Lucene-Solr-master-Solaris/lucene/common-build.xml:984: There were test failures: 613 suites (10 ignored), 2573 tests, 1 error, 87 ignored (72 assumptions) [seed: C556AE78FEB1CE6E]

Total time: 84 minutes 29 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-master-Solaris (64bit/jdk1.8.0) - Build # 638 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-master-Solaris/638/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseG1GC

No tests ran.

Build Log:
[...truncated 11409 lines...]
ERROR: Connection was broken: java.io.IOException: Unexpected termination of the channel
	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:50)
Caused by: java.io.EOFException
	at java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2325)
	at java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:2794)
	at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:801)
	at java.io.ObjectInputStream.<init>(ObjectInputStream.java:299)
	at hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:48)
	at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
	at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)

Build step 'Invoke Ant' marked build as failure
ERROR: Step ‘Archive the artifacts’ failed: no workspace for Lucene-Solr-master-Solaris #638
ERROR: Step ‘Scan for compiler warnings’ failed: no workspace for Lucene-Solr-master-Solaris #638
ERROR: Step ‘Publish JUnit test result report’ failed: no workspace for Lucene-Solr-master-Solaris #638
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any