You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2014/08/16 09:51:27 UTC
[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.7.0_65) - Build #
11034 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/11034/
Java: 32bit/jdk1.7.0_65 -server -XX:+UseConcMarkSweepGC
3 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.rest.TestManagedResourceStorage
Error Message:
SolrCore.getOpenCount()==2
Stack Trace:
java.lang.RuntimeException: SolrCore.getOpenCount()==2
at __randomizedtesting.SeedInfo.seed([DE9F1CDE97BE8B4A]:0)
at org.apache.solr.util.TestHarness.close(TestHarness.java:332)
at org.apache.solr.SolrTestCaseJ4.deleteCore(SolrTestCaseJ4.java:633)
at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:184)
at sun.reflect.GeneratedMethodAccessor30.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:790)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
FAILED: junit.framework.TestSuite.org.apache.solr.rest.TestManagedResourceStorage
Error Message:
5 threads leaked from SUITE scope at org.apache.solr.rest.TestManagedResourceStorage: 1) Thread[id=7790, name=coreZkRegister-4442-thread-1, state=WAITING, group=TGRP-TestManagedResourceStorage] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) 2) Thread[id=7782, name=OverseerStateUpdate-92286183554482179-188.138.97.18:_-n_0000000000, state=TIMED_WAITING, group=Overseer state updater.] at java.lang.Object.wait(Native Method) at org.apache.solr.cloud.DistributedQueue$LatchChildWatcher.await(DistributedQueue.java:266) at org.apache.solr.cloud.DistributedQueue.getChildren(DistributedQueue.java:309) at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:582) at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:560) at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:221) at java.lang.Thread.run(Thread.java:745) 3) Thread[id=7788, name=Thread-3101, state=WAITING, group=TGRP-TestManagedResourceStorage] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) at org.apache.solr.core.CloserThread.run(CoreContainer.java:905) 4) Thread[id=7787, name=searcherExecutor-4448-thread-1, state=WAITING, group=TGRP-TestManagedResourceStorage] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) 5) Thread[id=7784, name=OverseerHdfsCoreFailoverThread-92286183554482179-188.138.97.18:_-n_0000000000, state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.] at java.lang.Thread.sleep(Native Method) at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:136) at java.lang.Thread.run(Thread.java:745)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 5 threads leaked from SUITE scope at org.apache.solr.rest.TestManagedResourceStorage:
1) Thread[id=7790, name=coreZkRegister-4442-thread-1, state=WAITING, group=TGRP-TestManagedResourceStorage]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2) Thread[id=7782, name=OverseerStateUpdate-92286183554482179-188.138.97.18:_-n_0000000000, state=TIMED_WAITING, group=Overseer state updater.]
at java.lang.Object.wait(Native Method)
at org.apache.solr.cloud.DistributedQueue$LatchChildWatcher.await(DistributedQueue.java:266)
at org.apache.solr.cloud.DistributedQueue.getChildren(DistributedQueue.java:309)
at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:582)
at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:560)
at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:221)
at java.lang.Thread.run(Thread.java:745)
3) Thread[id=7788, name=Thread-3101, state=WAITING, group=TGRP-TestManagedResourceStorage]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:503)
at org.apache.solr.core.CloserThread.run(CoreContainer.java:905)
4) Thread[id=7787, name=searcherExecutor-4448-thread-1, state=WAITING, group=TGRP-TestManagedResourceStorage]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
5) Thread[id=7784, name=OverseerHdfsCoreFailoverThread-92286183554482179-188.138.97.18:_-n_0000000000, state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.]
at java.lang.Thread.sleep(Native Method)
at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:136)
at java.lang.Thread.run(Thread.java:745)
at __randomizedtesting.SeedInfo.seed([DE9F1CDE97BE8B4A]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.rest.TestManagedResourceStorage
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=7787, name=searcherExecutor-4448-thread-1, state=WAITING, group=TGRP-TestManagedResourceStorage] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) 2) Thread[id=7784, name=OverseerHdfsCoreFailoverThread-92286183554482179-188.138.97.18:_-n_0000000000, state=RUNNABLE, group=Overseer Hdfs SolrCore Failover Thread.] at java.lang.Throwable.fillInStackTrace(Native Method) at java.lang.Throwable.fillInStackTrace(Throwable.java:783) at java.lang.Throwable.<init>(Throwable.java:265) at java.lang.Exception.<init>(Exception.java:66) at java.lang.InterruptedException.<init>(InterruptedException.java:67) at java.lang.Thread.sleep(Native Method) at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:136) at java.lang.Thread.run(Thread.java:745) 3) Thread[id=7790, name=coreZkRegister-4442-thread-1, state=WAITING, group=TGRP-TestManagedResourceStorage] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) 4) Thread[id=7788, name=Thread-3101, state=WAITING, group=TGRP-TestManagedResourceStorage] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) at org.apache.solr.core.CloserThread.run(CoreContainer.java:905)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=7787, name=searcherExecutor-4448-thread-1, state=WAITING, group=TGRP-TestManagedResourceStorage]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2) Thread[id=7784, name=OverseerHdfsCoreFailoverThread-92286183554482179-188.138.97.18:_-n_0000000000, state=RUNNABLE, group=Overseer Hdfs SolrCore Failover Thread.]
at java.lang.Throwable.fillInStackTrace(Native Method)
at java.lang.Throwable.fillInStackTrace(Throwable.java:783)
at java.lang.Throwable.<init>(Throwable.java:265)
at java.lang.Exception.<init>(Exception.java:66)
at java.lang.InterruptedException.<init>(InterruptedException.java:67)
at java.lang.Thread.sleep(Native Method)
at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:136)
at java.lang.Thread.run(Thread.java:745)
3) Thread[id=7790, name=coreZkRegister-4442-thread-1, state=WAITING, group=TGRP-TestManagedResourceStorage]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
4) Thread[id=7788, name=Thread-3101, state=WAITING, group=TGRP-TestManagedResourceStorage]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:503)
at org.apache.solr.core.CloserThread.run(CoreContainer.java:905)
at __randomizedtesting.SeedInfo.seed([DE9F1CDE97BE8B4A]:0)
Build Log:
[...truncated 12178 lines...]
[junit4] Suite: org.apache.solr.rest.TestManagedResourceStorage
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.rest.TestManagedResourceStorage-DE9F1CDE97BE8B4A-001/init-core-data-001
[junit4] 2> 3134118 T7768 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (true)
[junit4] 2> 3134118 T7768 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 3134119 T7769 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 3134219 T7768 oasc.ZkTestServer.run start zk server on port:54967
[junit4] 2> 3134220 T7768 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3134224 T7775 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@8f19f name:ZooKeeperConnection Watcher:127.0.0.1:54967 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3134225 T7768 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 3134225 T7768 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 3134228 T7768 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3134229 T7777 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b604f1 name:ZooKeeperConnection Watcher:127.0.0.1:54967/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3134229 T7768 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 3134230 T7768 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 3134231 T7768 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 3134232 T7768 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 3134234 T7768 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 3134235 T7768 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 3134236 T7768 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 3134238 T7768 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 3134238 T7768 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 3134240 T7768 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 3134240 T7768 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 3134246 T7768 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 3134247 T7768 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 3134249 T7768 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 3134249 T7768 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 3134251 T7768 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 3134252 T7768 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 3134255 T7768 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 3134256 T7768 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 3134258 T7768 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 3134258 T7768 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 3134261 T7768 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 3134261 T7768 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 3134263 T7768 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 3134264 T7768 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 3134266 T7768 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 3134283 T7768 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 3134311 T7768 oas.SolrTestCaseJ4.initCore ####initCore
[junit4] 2> 3134311 T7768 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/'
[junit4] 2> 3134352 T7768 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
[junit4] 2> 3134353 T7768 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
[junit4] 2> 3134353 T7768 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/lib/README' to classloader
[junit4] 2> 3134402 T7768 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
[junit4] 2> 3134439 T7768 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 3134440 T7768 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 3134446 T7768 oass.IndexSchema.readSchema [null] Schema name=test
[junit4] 2> 3134687 T7768 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 3134691 T7768 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 3134693 T7768 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 3134698 T7768 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 3134701 T7768 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 3134703 T7768 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 3134703 T7768 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 3134704 T7768 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 3134704 T7768 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 3134704 T7768 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 3134704 T7768 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 3134705 T7768 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 3134705 T7768 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr
[junit4] 2> 3134705 T7768 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/'
[junit4] 2> 3134737 T7768 oasc.CoreContainer.<init> New CoreContainer 30831653
[junit4] 2> 3134737 T7768 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/]
[junit4] 2> 3134738 T7768 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4] 2> 3134738 T7768 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
[junit4] 2> 3134738 T7768 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4] 2> 3134738 T7768 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 3134739 T7768 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 3134739 T7768 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 3134739 T7768 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 3134739 T7768 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 3134739 T7768 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 3134740 T7768 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=30000&connTimeout=30000&retry=false
[junit4] 2> 3134740 T7768 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 3134740 T7768 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 3134741 T7768 oasc.CoreContainer.load Host Name:
[junit4] 2> 3134741 T7768 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:54967/solr
[junit4] 2> 3134759 T7768 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 3134784 T7768 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3135183 T7779 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@634dfb name:ZooKeeperConnection Watcher:127.0.0.1:54967 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3135183 T7768 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 3135185 T7768 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 3135186 T7781 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5b4ee name:ZooKeeperConnection Watcher:127.0.0.1:54967/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 3135187 T7768 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 3135188 T7768 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 3135190 T7768 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 3135191 T7768 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 3135193 T7768 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 3135194 T7768 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 3135196 T7768 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 3135197 T7768 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/188.138.97.18:_
[junit4] 2> 3135197 T7768 oascc.SolrZkClient.makePath makePath: /live_nodes/188.138.97.18:_
[junit4] 2> 3135199 T7768 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 3135200 T7768 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 3135201 T7768 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 3135202 T7768 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 188.138.97.18:_
[junit4] 2> 3135202 T7768 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 3135203 T7768 oasc.Overseer.start Overseer (id=92286183554482179-188.138.97.18:_-n_0000000000) starting
[junit4] 2> 3135205 T7768 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 3135211 T7768 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 3135211 T7783 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 3135212 T7768 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 3135214 T7768 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 3135216 T7768 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 3135217 T7782 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 3135220 T7785 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 3135220 T7785 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 3135221 T7785 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 3135221 T7786 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3135223 T7782 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "base_url":"http://188.138.97.18:",
[junit4] 2> "state":"down",
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "core_node_name":"188.138.97.18:__collection1",
[junit4] 2> "operation":"state",
[junit4] 2> "node_name":"188.138.97.18:_",
[junit4] 2> "shard":"shard1"}
[junit4] 2> 3135224 T7786 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 3136221 T7785 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 3136222 T7785 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 3136222 T7785 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 3136223 T7785 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 3136223 T7785 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/'
[junit4] 2> 3136224 T7785 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
[junit4] 2> 3136224 T7785 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
[junit4] 2> 3136224 T7785 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/lib/README' to classloader
[junit4] 2> 3136262 T7785 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
[junit4] 2> 3136298 T7785 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 3136299 T7785 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 3136304 T7785 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 3136522 T7785 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 3136526 T7785 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 3136528 T7785 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 3136535 T7785 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 3136537 T7785 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 3136540 T7785 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 3136540 T7785 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 3136541 T7785 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 3136541 T7785 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 3136542 T7785 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 3136542 T7785 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 3136542 T7785 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 3136542 T7785 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 3136543 T7785 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.rest.TestManagedResourceStorage-DE9F1CDE97BE8B4A-001/init-core-data-001/
[junit4] 2> 3136543 T7785 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@12c6a2c
[junit4] 2> 3136543 T7785 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.rest.TestManagedResourceStorage-DE9F1CDE97BE8B4A-001/init-core-data-001
[junit4] 2> 3136544 T7785 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.rest.TestManagedResourceStorage-DE9F1CDE97BE8B4A-001/init-core-data-001/index/
[junit4] 2> 3136544 T7785 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.rest.TestManagedResourceStorage-DE9F1CDE97BE8B4A-001/init-core-data-001/index' doesn't exist. Creating new index...
[junit4] 2> 3136544 T7785 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.rest.TestManagedResourceStorage-DE9F1CDE97BE8B4A-001/init-core-data-001/index
[junit4] 2> 3136544 T7785 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=11, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3488579181571619]
[junit4] 2> 3136545 T7785 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@10a4d5d lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@126321)),segFN=segments_1,generation=1}
[junit4] 2> 3136545 T7785 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 3136546 T7785 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4] 2> 3136548 T7785 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 3136549 T7785 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 3136549 T7785 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe-allfields"
[junit4] 2> 3136549 T7785 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4] 2> 3136549 T7785 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 3136550 T7785 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 3136550 T7785 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "uniq-fields"
[junit4] 2> 3136550 T7785 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4] 2> 3136550 T7785 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 3136551 T7785 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 3136551 T7785 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 3136551 T7785 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 3136551 T7785 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3136552 T7785 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3136552 T7785 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3136552 T7785 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3136552 T7785 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 3136553 T7785 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 3136553 T7785 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 3136553 T7785 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 3136554 T7785 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4] 2> 3136554 T7785 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4] 2> 3136554 T7785 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 3136555 T7785 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 3136555 T7785 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 3136556 T7785 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4] 2> 3136556 T7785 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4] 2> 3136556 T7785 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4] 2> 3136556 T7785 oasc.RequestHandlers.initHandlersFromConfig WARN Multiple requestHandler registered to the same name: /update ignoring: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 3136557 T7785 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 3136557 T7785 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 3136557 T7785 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 3136558 T7785 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 3136558 T7785 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 3136558 T7785 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 3136559 T7785 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 3136559 T7785 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 3136559 T7785 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 3136560 T7785 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4] 2> 3136560 T7785 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4] 2> 3136563 T7785 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3136564 T7785 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3136565 T7785 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3136566 T7785 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3136567 T7785 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 3136568 T7785 oasc.SolrCore.initDeprecatedSupport WARN solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4] 2> 3136570 T7785 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 3136570 T7785 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 3136571 T7785 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=34, maxMergeAtOnceExplicit=29, maxMergedSegmentMB=19.759765625, floorSegmentMB=1.888671875, forceMergeDeletesPctAllowed=12.13314766766255, segmentsPerTier=35.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8024096030075857
[junit4] 2> 3136621 T7785 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@10a4d5d lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@126321)),segFN=segments_1,generation=1}
[junit4] 2> 3136622 T7785 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 3136622 T7785 oass.SolrIndexSearcher.<init> Opening Searcher@4ec913[collection1] main
[junit4] 2> 3136622 T7785 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 3136623 T7785 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 3136623 T7785 oasr.RestManager.init Initializing RestManager with initArgs: {storageIO=org.apache.solr.rest.ManagedResourceStorage$InMemoryStorageIO}
[junit4] 2> 3136624 T7785 oasr.ManagedResourceStorage.load Reading _rest_managed.json using InMemoryStorage
[junit4] 2> 3136624 T7785 oasr.ManagedResource.reloadFromStorage WARN No stored data found for /rest/managed
[junit4] 2> 3136624 T7785 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using InMemoryStorage
[junit4] 2> 3136624 T7785 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 3136624 T7785 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4] 2> 3136723 T7785 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4] 2> 3136826 T7787 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4ec913[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 3136832 T7785 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 3136833 T7790 oasc.ZkController.register Register replica - core:collection1 address:http://188.138.97.18: collection:collection1 shard:shard1
[junit4] 2> 3136834 T7790 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 3136835 T7768 oas.SolrTestCaseJ4.initCore ####initCore end
[junit4] 2> 3136838 T7790 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 3136842 T7786 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3136842 T7790 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 3136844 T7790 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> 3136844 T7790 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://188.138.97.18:/collection1/ shard1
[junit4] 2> 3136845 T7790 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 3136845 T7768 oas.SolrTestCaseJ4.setUp ###Starting testFileBasedJsonStorage
[junit4] 2> 3136846 T7768 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.rest.TestManagedResourceStorage-DE9F1CDE97BE8B4A-001/json-storage-001/'
[junit4] 2> 3136847 T7786 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 3136851 T7786 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 3136851 T7786 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 3136861 T7768 oasr.ManagedResourceStorage$FileStorageIO.configure File-based storage initialized to use dir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.rest.TestManagedResourceStorage-DE9F1CDE97BE8B4A-001/json-storage-001/managed
[junit4] 2> 3136862 T7768 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _test_foo.json using file:dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.rest.TestManagedResourceStorage-DE9F1CDE97BE8B4A-001/json-storage-001/managed
[junit4] 2> 3136862 T7768 oasr.ManagedResourceStorage.load Reading _test_foo.json using file:dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.rest.TestManagedResourceStorage-DE9F1CDE97BE8B4A-001/json-storage-001/managed
[junit4] 2> 3136863 T7768 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _test_foo.json using file:dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.rest.TestManagedResourceStorage-DE9F1CDE97BE8B4A-001/json-storage-001/managed
[junit4] 2> 3136863 T7768 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _test_foo.json using file:dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.rest.TestManagedResourceStorage-DE9F1CDE97BE8B4A-001/json-storage-001/managed
[junit4] 2> 3136863 T7768 oasr.ManagedResourceStorage.load Reading _test_foo.json using file:dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.rest.TestManagedResourceStorage-DE9F1CDE97BE8B4A-001/json-storage-001/managed
[junit4] 2> 3136864 T7768 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _test_foo.json using file:dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.rest.TestManagedResourceStorage-DE9F1CDE97BE8B4A-001/json-storage-001/managed
[junit4] 2> 3136864 T7768 oas.SolrTestCaseJ4.tearDown ###Ending testFileBasedJsonStorage
[junit4] 2> 3136871 T7768 oas.SolrTestCaseJ4.setUp ###Starting testZkBasedJsonStorage
[junit4] 2> 3136871 T7768 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './'
[junit4] 2> 3136883 T7768 oascc.SolrZkClient.makePath makePath: /test
[junit4] 2> 3136884 T7768 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /test
[junit4] 2> 3136885 T7768 oascc.SolrZkClient.makePath makePath: /test/_test_foo.json
[junit4] 2> 3136886 T7768 oasr.ManagedResourceStorage$ZooKeeperStorageIO$1.close Wrote 136 bytes to new znode /test/_test_foo.json
[junit4] 2> 3136887 T7768 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _test_foo.json using ZooKeeperStorageIO:path=/test
[junit4] 2> 3136887 T7768 oasr.ManagedResourceStorage.load Reading _test_foo.json using ZooKeeperStorageIO:path=/test
[junit4] 2> 3136888 T7768 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 136 bytes from znode /test/_test_foo.json
[junit4] 2> 3136888 T7768 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _test_foo.json using ZooKeeperStorageIO:path=/test
[junit4] 2> 3136889 T7768 oasr.ManagedResourceStorage$ZooKeeperStorageIO$1.close Wrote 175 bytes to existing znode /test/_test_foo.json
[junit4] 2> 3136889 T7768 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _test_foo.json using ZooKeeperStorageIO:path=/test
[junit4] 2> 3136890 T7768 oasr.ManagedResourceStorage.load Reading _test_foo.json using ZooKeeperStorageIO:path=/test
[junit4] 2> 3136890 T7768 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 175 bytes from znode /test/_test_foo.json
[junit4] 2> 3136891 T7768 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _test_foo.json using ZooKeeperStorageIO:path=/test
[junit4] 2> 3136891 T7786 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 3136892 T7768 oas.SolrTestCaseJ4.tearDown ###Ending testZkBasedJsonStorage
[junit4] 2> 3136893 T7768 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:54967 54967
[junit4] 2> 3136899 T7790 oasc.ZkController.register We are http://188.138.97.18:/collection1/ and leader is http://188.138.97.18:/collection1/
[junit4] 2> 3137215 T7783 oasc.OverseerCollectionProcessor.amILeader org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.cloud.OverseerCollectionProcessor.amILeader(OverseerCollectionProcessor.java:516)
[junit4] 2> at org.apache.solr.cloud.OverseerCollectionProcessor.run(OverseerCollectionProcessor.java:271)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 3137436 T7769 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:54967 54967
[junit4] 2> 3137435 T7790 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://188.138.97.18:
[junit4] 2> 3137437 T7790 oasc.ZkController.checkRecovery WARN Skipping recovery according to sys prop solrcloud.skip.autorecovery
[junit4] 2> 3137438 T7768 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> 3137437 T7783 oasc.OverseerCollectionProcessor.amILeader According to ZK I (id=92286183554482179-188.138.97.18:_-n_0000000000) am no longer a leader.
[junit4] 2> 3137438 T7790 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 3137439 T7790 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 3137439 T7790 oasc.ZkController.getLeaderInitiatedRecoveryState WARN Unable to read /collections/collection1/leader_initiated_recovery/shard1/188.138.97.18:__collection1 due to: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_initiated_recovery/shard1/188.138.97.18:__collection1
[junit4] 2> 3137440 T7790 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 3137440 T7790 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 3137440 T7790 oasc.ZkContainer$2.run ERROR org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer/queue/qn-
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:273)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:270)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:270)
[junit4] 2> at org.apache.solr.cloud.DistributedQueue.createData(DistributedQueue.java:369)
[junit4] 2> at org.apache.solr.cloud.DistributedQueue.offer(DistributedQueue.java:353)
[junit4] 2> at org.apache.solr.cloud.ZkController.publish(ZkController.java:1120)
[junit4] 2> at org.apache.solr.cloud.ZkController.publish(ZkController.java:1038)
[junit4] 2> at org.apache.solr.cloud.ZkController.publish(ZkController.java:1034)
[junit4] 2> at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:207)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 3137441 T7790 oasc.SolrException.log ERROR :org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer/queue/qn-
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:273)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:270)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:270)
[junit4] 2> at org.apache.solr.cloud.DistributedQueue.createData(DistributedQueue.java:369)
[junit4] 2> at org.apache.solr.cloud.DistributedQueue.offer(DistributedQueue.java:353)
[junit4] 2> at org.apache.solr.cloud.ZkController.publish(ZkController.java:1120)
[junit4] 2> at org.apache.solr.cloud.ZkController.publish(ZkController.java:1038)
[junit4] 2> at org.apache.solr.cloud.ZkController.publish(ZkController.java:1034)
[junit4] 2> at org.apache.solr.cloud.ZkController.register(ZkController.java:849)
[junit4] 2> at org.apache.solr.cloud.ZkController.register(ZkController.java:763)
[junit4] 2> at org.apache.solr.core.ZkContainer$2.run(ZkContainer.java:200)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 3894 T7767 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 5 leaked thread(s).
[junit4] 2> 24102 T7767 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 5 threads leaked from SUITE scope at org.apache.solr.rest.TestManagedResourceStorage:
[junit4] 2> 1) Thread[id=7790, name=coreZkRegister-4442-thread-1, state=WAITING, group=TGRP-TestManagedResourceStorage]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 2) Thread[id=7782, name=OverseerStateUpdate-92286183554482179-188.138.97.18:_-n_0000000000, state=TIMED_WAITING, group=Overseer state updater.]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at org.apache.solr.cloud.DistributedQueue$LatchChildWatcher.await(DistributedQueue.java:266)
[junit4] 2> at org.apache.solr.cloud.DistributedQueue.getChildren(DistributedQueue.java:309)
[junit4] 2> at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:582)
[junit4] 2> at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:560)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:221)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 3) Thread[id=7788, name=Thread-3101, state=WAITING, group=TGRP-TestManagedResourceStorage]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Object.wait(Object.java:503)
[junit4] 2> at org.apache.solr.core.CloserThread.run(CoreContainer.java:905)
[junit4] 2> 4) Thread[id=7787, name=searcherExecutor-4448-thread-1, state=WAITING, group=TGRP-TestManagedResourceStorage]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 5) Thread[id=7784, name=OverseerHdfsCoreFailoverThread-92286183554482179-188.138.97.18:_-n_0000000000, state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.]
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:136)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 24103 T7767 ccr.ThreadLeakControl.tryToInterruptAll Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=7790, name=coreZkRegister-4442-thread-1, state=WAITING, group=TGRP-TestManagedResourceStorage]
[junit4] 2> 2) Thread[id=7782, name=OverseerStateUpdate-92286183554482179-188.138.97.18:_-n_0000000000, state=TIMED_WAITING, group=Overseer state updater.]
[junit4] 2> 3) Thread[id=7788, name=Thread-3101, state=WAITING, group=TGRP-TestManagedResourceStorage]
[junit4] 2> 4) Thread[id=7787, name=searcherExecutor-4448-thread-1, state=WAITING, group=TGRP-TestManagedResourceStorage]
[junit4] 2> 5) Thread[id=7784, name=OverseerHdfsCoreFailoverThread-92286183554482179-188.138.97.18:_-n_0000000000, state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.]
[junit4] 2> 3158221 T7782 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 188.138.97.18:_
[junit4] 2> 3158221 T7791 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:321)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:89)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:293)
[junit4] 2>
[junit4] 2> 27126 T7767 ccr.ThreadLeakControl.tryToInterruptAll SEVERE There are still zombie threads that couldn't be terminated:
[junit4] 2> 1) Thread[id=7787, name=searcherExecutor-4448-thread-1, state=WAITING, group=TGRP-TestManagedResourceStorage]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 2) Thread[id=7784, name=OverseerHdfsCoreFailoverThread-92286183554482179-188.138.97.18:_-n_0000000000, state=RUNNABLE, group=Overseer Hdfs SolrCore Failover Thread.]
[junit4] 2> at java.lang.Throwable.fillInStackTrace(Native Method)
[junit4] 2> at java.lang.Throwable.fillInStackTrace(Throwable.java:783)
[junit4] 2> at java.lang.Throwable.<init>(Throwable.java:265)
[junit4] 2> at java.lang.Exception.<init>(Exception.java:66)
[junit4] 2> at java.lang.InterruptedException.<init>(InterruptedException.java:67)
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:136)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 3) Thread[id=7790, name=coreZkRegister-4442-thread-1, state=WAITING, group=TGRP-TestManagedResourceStorage]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 4) Thread[id=7788, name=Thread-3101, state=WAITING, group=TGRP-TestManagedResourceStorage]
[junit4] 2> at java.lang.Object.wait(Native Method)
[junit4] 2> at java.lang.Object.wait(Object.java:503)
[junit4] 2> at org.apache.solr.core.CloserThread.run(CoreContainer.java:905)
[junit4] 2> NOTE: test params are: codec=Lucene410: {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=lt, timezone=Asia/Aqtau
[junit4] 2> NOTE: Linux 3.13.0-32-generic i386/Oracle Corporation 1.7.0_65 (32-bit)/cpus=8,threads=5,free=115833056,total=335560704
[junit4] 2> NOTE: All tests run in this JVM: [TestIndexSearcher, TestRealTimeGet, TestOmitPositions, TestRandomFaceting, HardAutoCommitTest, SolrRequestParserTest, FullSolrCloudDistribCmdsTest, SuggesterFSTTest, AnalysisErrorHandlingTest, TestCollationFieldDocValues, TestIBSimilarityFactory, DistribDocExpirationUpdateProcessorTest, CurrencyFieldXmlFileTest, TestSolrXml, RAMDirectoryFactoryTest, TestQueryUtils, TestCoreContainer, FileBasedSpellCheckerTest, ChaosMonkeySafeLeaderTest, TestOverriddenPrefixQueryForCustomFieldType, TestPostingsSolrHighlighter, BasicDistributedZk2Test, TestUtils, TestRequestStatusCollectionAPI, DistributedFacetPivotLongTailTest, NumericFieldsTest, TestValueSourceCache, TestCollectionAPI, TestXIncludeConfig, MigrateRouteKeyTest, ChangedSchemaMergeTest, AssignTest, TestLeaderElectionZkExpiry, TestOrdValues, SimplePostToolTest, SolrXmlInZkTest, HdfsCollectionsAPIDistributedZkTest, CoreContainerCoreInitFailuresTest, PolyFieldTest, AddSchemaFieldsUpdateProcessorFactoryTest, AbstractAnalyticsFacetTest, TestFileDictionaryLookup, DistanceFunctionTest, SolrCoreTest, SliceStateTest, DistributedQueryComponentCustomSortTest, TestPhraseSuggestions, TestZkChroot, ClusterStateTest, TestCloudManagedSchemaConcurrent, OverseerRolesTest, RemoteQueryErrorTest, FieldFacetTest, QueryParsingTest, DistributedQueryElevationComponentTest, MultiTermTest, TestJoin, HdfsBasicDistributedZk2Test, TestAddFieldRealTimeGet, TestBM25SimilarityFactory, BasicZkTest, TestUpdate, HdfsWriteToMultipleCollectionsTest, ScriptEngineTest, RequestHandlersTest, TestCloudInspectUtil, TestPHPSerializedResponseWriter, AlternateDirectoryTest, DirectSolrConnectionTest, TestQuerySenderNoQuery, TestHighFrequencyDictionaryFactory, SOLR749Test, SpatialFilterTest, TestHighlightDedupGrouping, ShardRoutingCustomTest, SolrIndexConfigTest, URLClassifyProcessorTest, LukeRequestHandlerTest, TestRecovery, TestCharFilters, TestSolrQueryParser, TestCopyFieldCollectionResource, TestExceedMaxTermLength, SolrPluginUtilsTest, DistributedTermsComponentTest, TestGroupingSearch, TestInitQParser, LegacyHTMLStripCharFilterTest, SynonymTokenizerTest, TestCSVLoader, TestLFUCache, LeaderElectionIntegrationTest, TestPivotHelperCode, AnalyticsMergeStrategyTest, TestDynamicFieldCollectionResource, ResourceLoaderTest, TestDistributedGrouping, TestPseudoReturnFields, TestSolrQueryParserDefaultOperatorResource, DateFieldTest, ZkSolrClientTest, ShardRoutingTest, TestRandomDVFaceting, ZkCLITest, TestDistributedSearch, AutoCommitTest, TestHashPartitioner, TermVectorComponentDistributedTest, ZkControllerTest, TestStressReorder, StatsComponentTest, SpellCheckComponentTest, PeerSyncTest, TestFunctionQuery, TestLazyCores, TestBadConfig, ShowFileRequestHandlerTest, SolrIndexSplitterTest, SignatureUpdateProcessorFactoryTest, DocValuesTest, SuggesterTSTTest, NoCacheHeaderTest, DirectUpdateHandlerOptimizeTest, DefaultValueUpdateProcessorTest, SortByFunctionTest, TestSolrDeletionPolicy1, SolrInfoMBeanTest, XsltUpdateRequestHandlerTest, DebugComponentTest, IndexBasedSpellCheckerTest, StandardRequestHandlerTest, TestWriterPerf, TestReversedWildcardFilterFactory, TestQueryTypes, DocumentBuilderTest, TermVectorComponentTest, FieldAnalysisRequestHandlerTest, FastVectorHighlighterTest, LoggingHandlerTest, TestJmxIntegration, ReturnFieldsTest, JSONWriterTest, MBeansHandlerTest, JsonLoaderTest, CSVRequestHandlerTest, TestQuerySenderListener, BadComponentTest, TestMergePolicyConfig, TestSolrDeletionPolicy2, SampleTest, TestBinaryField, TestElisionMultitermQuery, OutputWriterTest, ExternalFileFieldSortTest, TestSolrCoreProperties, TestLuceneMatchVersion, EchoParamsTest, TestSweetSpotSimilarityFactory, TestPerFieldSimilarity, TestDefaultSimilarityFactory, TestFastWriter, TestJmxMonitoredMap, TestFastOutputStream, PluginInfoTest, TestFastLRUCache, DateMathParserTest, PreAnalyzedFieldTest, TestSystemIdResolver, PrimUtilsTest, TestSolrJ, TestDocumentBuilder, ZkNodePropsTest, UUIDFieldTest, FileUtilsTest, CircularListTest, TestRTGBase, CursorPagingTest, SolrTestCaseJ4Test, TestDistributedMissingSort, AbstractAnalyticsStatsTest, NoFacetTest, ExpressionTest, QueryFacetTest, FunctionTest, AliasIntegrationTest, DeleteReplicaTest, DeleteShardTest, MultiThreadedOCPTest, RollingRestartTest, SharedFSAutoReplicaFailoverUtilsTest, TestCloudPivotFacet, TestDistribDocBasedVersion, TestMiniSolrCloudCluster, HdfsBasicDistributedZkTest, HdfsRecoveryZkTest, HdfsSyncSliceTest, HdfsUnloadDistributedZkTest, StressHdfsTest, QueryResultKeyTest, TestImplicitCoreProperties, TestInfoStreamLogging, TestNonNRTOpen, TestReloadAndDeleteDocs, TestSolrXmlPersistence, TestReplicationHandlerBackup, CoreAdminRequestStatusTest, CoreMergeIndexesAdminHandlerTest, InfoHandlerTest, DistributedDebugComponentTest, DistributedExpandComponentTest, DistributedFacetPivotLargeTest, DistributedSuggestComponentTest, TestChildDocTransformer, TestSortingResponseWriter, TestManagedResource, TestManagedResourceStorage]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestManagedResourceStorage -Dtests.seed=DE9F1CDE97BE8B4A -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=lt -Dtests.timezone=Asia/Aqtau -Dtests.file.encoding=US-ASCII
[junit4] ERROR 0.00s J1 | TestManagedResourceStorage (suite) <<<
[junit4] > Throwable #1: java.lang.RuntimeException: SolrCore.getOpenCount()==2
[junit4] > at __randomizedtesting.SeedInfo.seed([DE9F1CDE97BE8B4A]:0)
[junit4] > at org.apache.solr.util.TestHarness.close(TestHarness.java:332)
[junit4] > at org.apache.solr.SolrTestCaseJ4.deleteCore(SolrTestCaseJ4.java:633)
[junit4] > at org.apache.solr.SolrTestCaseJ4.afterClass(SolrTestCaseJ4.java:184)
[junit4] > at java.lang.Thread.run(Thread.java:745)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: 5 threads leaked from SUITE scope at org.apache.solr.rest.TestManagedResourceStorage:
[junit4] > 1) Thread[id=7790, name=coreZkRegister-4442-thread-1, state=WAITING, group=TGRP-TestManagedResourceStorage]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 2) Thread[id=7782, name=OverseerStateUpdate-92286183554482179-188.138.97.18:_-n_0000000000, state=TIMED_WAITING, group=Overseer state updater.]
[junit4] > at java.lang.Object.wait(Native Method)
[junit4] > at org.apache.solr.cloud.DistributedQueue$LatchChildWatcher.await(DistributedQueue.java:266)
[junit4] > at org.apache.solr.cloud.DistributedQueue.getChildren(DistributedQueue.java:309)
[junit4] > at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:582)
[junit4] > at org.apache.solr.cloud.DistributedQueue.peek(DistributedQueue.java:560)
[junit4] > at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:221)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 3) Thread[id=7788, name=Thread-3101, state=WAITING, group=TGRP-TestManagedResourceStorage]
[junit4] > at java.lang.Object.wait(Native Method)
[junit4] > at java.lang.Object.wait(Object.java:503)
[junit4] > at org.apache.solr.core.CloserThread.run(CoreContainer.java:905)
[junit4] > 4) Thread[id=7787, name=searcherExecutor-4448-thread-1, state=WAITING, group=TGRP-TestManagedResourceStorage]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 5) Thread[id=7784, name=OverseerHdfsCoreFailoverThread-92286183554482179-188.138.97.18:_-n_0000000000, state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:136)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > at __randomizedtesting.SeedInfo.seed([DE9F1CDE97BE8B4A]:0)Throwable #3: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=7787, name=searcherExecutor-4448-thread-1, state=WAITING, group=TGRP-TestManagedResourceStorage]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 2) Thread[id=7784, name=OverseerHdfsCoreFailoverThread-92286183554482179-188.138.97.18:_-n_0000000000, state=RUNNABLE, group=Overseer Hdfs SolrCore Failover Thread.]
[junit4] > at java.lang.Throwable.fillInStackTrace(Native Method)
[junit4] > at java.lang.Throwable.fillInStackTrace(Throwable.java:783)
[junit4] > at java.lang.Throwable.<init>(Throwable.java:265)
[junit4] > at java.lang.Exception.<init>(Exception.java:66)
[junit4] > at java.lang.InterruptedException.<init>(InterruptedException.java:67)
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:136)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 3) Thread[id=7790, name=coreZkRegister-4442-thread-1, state=WAITING, group=TGRP-TestManagedResourceStorage]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] > 4) Thread[id=7788, name=Thread-3101, state=WAITING, group=TGRP-TestManagedResourceStorage]
[junit4] > at java.lang.Object.wait(Native Method)
[junit4] > at java.lang.Object.wait(Object.java:503)
[junit4] > at org.apache.solr.core.CloserThread.run(CoreContainer.java:905)
[junit4] > at __randomizedtesting.SeedInfo.seed([DE9F1CDE97BE8B4A]:0)
[junit4] Completed on J1 in 27.24s, 2 tests, 3 errors <<< FAILURES!
[...truncated 124 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:474: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:454: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:496: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1350: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:965: There were test failures: 430 suites, 1782 tests, 3 suite-level errors, 135 ignored (19 assumptions)
Total time: 99 minutes 55 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 32bit/jdk1.7.0_65 -server -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-trunk-Linux (32bit/ibm-j9-jdk7) - Build #
11039 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/11039/
Java: 32bit/ibm-j9-jdk7 -Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;}
1 tests failed.
REGRESSION: org.apache.solr.update.HardAutoCommitTest.testCommitWithin
Error Message:
Exception during query
Stack Trace:
java.lang.RuntimeException: Exception during query
at __randomizedtesting.SeedInfo.seed([9057D8056DEB331C:2A85B77DEEC5DD09]:0)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:722)
at org.apache.solr.update.HardAutoCommitTest.testCommitWithin(HardAutoCommitTest.java:79)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:94)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:619)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:853)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=0]
xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">2526</int></lst><result name="response" numFound="1" start="0"><doc><int name="id">529</int><arr name="range_facet_l"><long>529</long></arr><arr name="field_t"><str>what's inside?</str></arr><arr name="subject"><str>info</str></arr><long name="_version_">1476640896451608576</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2014-08-17T00:14:57.501Z</date></doc></result>
</response>
request was:start=0&rows=20&version=2.2&qt=standard&q=id:529
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:715)
... 41 more
Build Log:
[...truncated 11489 lines...]
[junit4] Suite: org.apache.solr.update.HardAutoCommitTest
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.update.HardAutoCommitTest-9057D8056DEB331C-001/init-core-data-001
[junit4] 2> 2011049 T5446 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false)
[junit4] 2> 2011049 T5446 oas.SolrTestCaseJ4.initCore ####initCore
[junit4] 2> 2011050 T5446 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/'
[junit4] 2> 2011050 T5446 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
[junit4] 2> 2011051 T5446 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
[junit4] 2> 2011051 T5446 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/lib/README' to classloader
[junit4] 2> 2011089 T5446 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
[junit4] 2> 2011142 T5446 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 2011143 T5446 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 2011148 T5446 oass.IndexSchema.readSchema [null] Schema name=test
[junit4] 2> 2011455 T5446 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 2011460 T5446 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 2011463 T5446 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 2011468 T5446 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2011472 T5446 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2011475 T5446 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 2011476 T5446 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 2011476 T5446 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 2011477 T5446 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 2011477 T5446 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 2011477 T5446 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 2011478 T5446 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 2011478 T5446 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr
[junit4] 2> 2011479 T5446 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/'
[junit4] 2> 2011521 T5446 oasc.CoreContainer.<init> New CoreContainer -438346413
[junit4] 2> 2011521 T5446 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/]
[junit4] 2> 2011522 T5446 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4] 2> 2011523 T5446 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
[junit4] 2> 2011523 T5446 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4] 2> 2011524 T5446 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 2011524 T5446 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 2011524 T5446 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 2011525 T5446 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 2011525 T5446 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 2011525 T5446 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 2011527 T5446 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=30000&connTimeout=30000&retry=false
[junit4] 2> 2011528 T5446 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 2011529 T5446 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 2011529 T5446 oasc.CoreContainer.load Host Name:
[junit4] 2> 2011533 T5447 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/'
[junit4] 2> 2011533 T5447 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
[junit4] 2> 2011534 T5447 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
[junit4] 2> 2011534 T5447 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/lib/README' to classloader
[junit4] 2> 2011575 T5447 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
[junit4] 2> 2011624 T5447 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 2011624 T5447 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 2011630 T5447 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 2011929 T5447 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 2011938 T5447 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 2011941 T5447 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 2011948 T5447 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2011951 T5447 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2011955 T5447 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 2011955 T5447 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 2011955 T5447 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 2011956 T5447 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 2011956 T5447 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 2011957 T5447 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 2011957 T5447 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from instancedir /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/
[junit4] 2> 2011957 T5447 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4] 2> 2011958 T5447 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.update.HardAutoCommitTest-9057D8056DEB331C-001/init-core-data-001/
[junit4] 2> 2011958 T5447 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@a86ada0a
[junit4] 2> 2011959 T5447 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.update.HardAutoCommitTest-9057D8056DEB331C-001/init-core-data-001
[junit4] 2> 2011959 T5447 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.update.HardAutoCommitTest-9057D8056DEB331C-001/init-core-data-001/index/
[junit4] 2> 2011959 T5447 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.update.HardAutoCommitTest-9057D8056DEB331C-001/init-core-data-001/index' doesn't exist. Creating new index...
[junit4] 2> 2011960 T5447 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.update.HardAutoCommitTest-9057D8056DEB331C-001/init-core-data-001/index
[junit4] 2> 2011960 T5447 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=22, maxMergeAtOnceExplicit=20, maxMergedSegmentMB=12.333984375, floorSegmentMB=0.927734375, forceMergeDeletesPctAllowed=16.56479180823275, segmentsPerTier=29.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8136037390990376
[junit4] 2> 2011961 T5447 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@22b1df63 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@b8ac9805)),segFN=segments_1,generation=1}
[junit4] 2> 2011961 T5447 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 2011962 T5447 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4] 2> 2011964 T5447 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 2011965 T5447 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 2011965 T5447 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe-allfields"
[junit4] 2> 2011965 T5447 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4] 2> 2011965 T5447 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 2011966 T5447 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 2011966 T5447 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "uniq-fields"
[junit4] 2> 2011966 T5447 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4] 2> 2011967 T5447 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 2011967 T5447 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 2011967 T5447 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 2011968 T5447 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 2011968 T5447 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 2011968 T5447 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 2011969 T5447 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 2011969 T5447 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 2011969 T5447 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 2011970 T5447 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 2011970 T5447 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 2011970 T5447 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 2011971 T5447 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4] 2> 2011971 T5447 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4] 2> 2011971 T5447 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 2011972 T5447 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 2011972 T5447 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 2011973 T5447 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4] 2> 2011973 T5447 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4] 2> 2011973 T5447 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4] 2> 2011973 T5447 oasc.RequestHandlers.initHandlersFromConfig WARN Multiple requestHandler registered to the same name: /update ignoring: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 2011974 T5447 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 2011974 T5447 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 2011974 T5447 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 2011975 T5447 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 2011975 T5447 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 2011976 T5447 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 2011976 T5447 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 2011976 T5447 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 2011977 T5447 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 2011977 T5447 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4] 2> 2011977 T5447 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4] 2> 2011979 T5447 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2011980 T5447 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2011981 T5447 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2011982 T5447 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2011983 T5447 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2011984 T5447 oasc.SolrCore.initDeprecatedSupport WARN solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4] 2> 2011986 T5447 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 2011987 T5447 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 2011987 T5447 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=43, maxMergeAtOnceExplicit=36, maxMergedSegmentMB=37.1474609375, floorSegmentMB=0.7314453125, forceMergeDeletesPctAllowed=13.36498295451388, segmentsPerTier=16.0, maxCFSSegmentSizeMB=1.1298828125, noCFSRatio=0.0
[junit4] 2> 2011988 T5447 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@22b1df63 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@b8ac9805)),segFN=segments_1,generation=1}
[junit4] 2> 2011988 T5447 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 2011988 T5447 oass.SolrIndexSearcher.<init> Opening Searcher@f83d332e[collection1] main
[junit4] 2> 2011989 T5447 oasr.RestManager.init Initializing RestManager with initArgs: {storageIO=org.apache.solr.rest.ManagedResourceStorage$InMemoryStorageIO}
[junit4] 2> 2011989 T5447 oasr.ManagedResourceStorage.load Reading _rest_managed.json using InMemoryStorage
[junit4] 2> 2011989 T5447 oasr.ManagedResource.reloadFromStorage WARN No stored data found for /rest/managed
[junit4] 2> 2011990 T5447 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using InMemoryStorage
[junit4] 2> 2011990 T5447 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 2011990 T5447 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4] 2> 2012213 T5447 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4] 2> 2012233 T5448 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@f83d332e[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 2012236 T5447 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 2012238 T5446 oas.SolrTestCaseJ4.initCore ####initCore end
[junit4] 2> 2012242 T5446 oas.SolrTestCaseJ4.setUp ###Starting testCommitWithin
[junit4] 2> ASYNC NEW_CORE C1235 name=collection1 org.apache.solr.core.SolrCore@d85e759b
[junit4] 2> 2012243 T5446 C1235 oasup.LogUpdateProcessor.finish [collection1] webapp=null path=null params={} {deleteByQuery=*:* (-1476640895297126400)} 0 0
[junit4] 2> 2012247 T5446 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/'
[junit4] 2> 2012248 T5446 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
[junit4] 2> 2012249 T5446 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
[junit4] 2> 2012249 T5446 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/lib/README' to classloader
[junit4] 2> 2012290 T5446 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
[junit4] 2> 2012344 T5446 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 2012345 T5446 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 2012352 T5446 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 2012675 T5446 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 2012681 T5446 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 2012684 T5446 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 2012690 T5446 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2012694 T5446 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2012697 T5446 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 2012698 T5446 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 2012698 T5446 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 2012699 T5446 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 2012700 T5446 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 2012700 T5446 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 2012700 T5446 oasc.CoreContainer.reload Reloading SolrCore 'collection1' using configuration from instancedir /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/
[junit4] 2> 2012701 T5446 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.update.HardAutoCommitTest-9057D8056DEB331C-001/init-core-data-001/
[junit4] 2> 2012701 T5446 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@a86ada0a
[junit4] 2> 2012702 T5446 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.update.HardAutoCommitTest-9057D8056DEB331C-001/init-core-data-001/index/
[junit4] 2> 2012703 T5446 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4] 2> 2012706 T5446 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 2012706 T5446 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 2012707 T5446 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe-allfields"
[junit4] 2> 2012707 T5446 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4] 2> 2012707 T5446 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 2012708 T5446 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 2012708 T5446 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "uniq-fields"
[junit4] 2> 2012709 T5446 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4] 2> 2012709 T5446 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 2012710 T5446 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 2012710 T5446 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 2012711 T5446 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 2012712 T5446 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 2012712 T5446 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 2012713 T5446 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 2012713 T5446 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 2012714 T5446 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 2012714 T5446 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 2012715 T5446 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 2012715 T5446 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 2012716 T5446 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4] 2> 2012717 T5446 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4] 2> 2012717 T5446 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 2012718 T5446 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 2012718 T5446 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 2012719 T5446 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4] 2> 2012719 T5446 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4] 2> 2012720 T5446 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4] 2> 2012720 T5446 oasc.RequestHandlers.initHandlersFromConfig WARN Multiple requestHandler registered to the same name: /update ignoring: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 2012721 T5446 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 2012721 T5446 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 2012722 T5446 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 2012722 T5446 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 2012723 T5446 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 2012724 T5446 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 2012724 T5446 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 2012725 T5446 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 2012725 T5446 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4] 2> 2012726 T5446 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4] 2> 2012727 T5446 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4] 2> 2012728 T5446 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2012729 T5446 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2012731 T5446 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2012732 T5446 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2012733 T5446 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2012735 T5446 oasc.SolrCore.initDeprecatedSupport WARN solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4] 2> 2012736 T5446 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 2012737 T5446 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 2012738 T5446 oass.SolrIndexSearcher.<init> Opening Searcher@c1c1d8d9[collection1] main
[junit4] 2> 2012738 T5446 oasr.RestManager.init Initializing RestManager with initArgs: {storageIO=org.apache.solr.rest.ManagedResourceStorage$InMemoryStorageIO}
[junit4] 2> 2012739 T5446 oasr.ManagedResourceStorage.load Reading _rest_managed.json using InMemoryStorage
[junit4] 2> 2012739 T5446 oasr.ManagedResource.reloadFromStorage WARN No stored data found for /rest/managed
[junit4] 2> 2012740 T5446 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using InMemoryStorage
[junit4] 2> 2012740 T5446 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 2012740 T5446 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4] 2> 2012742 T5446 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4] 2> 2012748 T5450 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@c1c1d8d9[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 2012753 T5446 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4] 2> 2012754 T5446 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4] 2> 2012754 T5446 oasu.DefaultSolrCoreState.newIndexWriter Closing old IndexWriter... core=collection1
[junit4] 2> 2012755 T5446 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=10, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8665987381791024]
[junit4] 2> 2012755 T5446 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@22b1df63 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@b8ac9805)),segFN=segments_1,generation=1}
[junit4] 2> 2012756 T5446 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 2012756 T5446 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4] 2> 2012757 T5446 oass.SolrIndexSearcher.<init> Opening Searcher@6b5be41f[collection1] main
[junit4] 2> 2012757 T5446 oasc.CoreContainer.registerCore replacing core: collection1
[junit4] 2> 2012758 T5446 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@d85e759b
[junit4] 2> 2012758 T5450 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6b5be41f[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 2012763 T5446 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=1,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=0}
[junit4] 2> 2012763 T5446 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 2013345 T5446 oasup.LogUpdateProcessor.finish [collection1] {add=[529 (1476640896451608576)]} 0 575
[junit4] 2> 2015345 T5451 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2015349 T5451 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4] 2> commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@22b1df63 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@b8ac9805)),segFN=segments_1,generation=1}
[junit4] 2> commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@22b1df63 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@b8ac9805)),segFN=segments_2,generation=2}
[junit4] 2> 2015350 T5451 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4] 2> 2015351 T5451 oass.SolrIndexSearcher.<init> Opening Searcher@d02de2ea[collection1] main
[junit4] 2> 2015352 T5451 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4] 2> 2015353 T5450 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@d02de2ea[collection1] main{UninvertingDirectoryReader(Uninverting(_0(5.0.0):C1))}
[junit4] 2> ASYNC NEW_CORE C1236 name=collection1 org.apache.solr.core.SolrCore@ee2e94da
[junit4] 2> 2015872 T5446 C1236 oasc.SolrCore.execute [collection1] webapp=null path=null params={start=0&rows=20&version=2.2&qt=standard&q=id:529} hits=1 status=0 QTime=2526
[junit4] 2> 2015926 T5446 oas.SolrTestCaseJ4.assertQ ERROR REQUEST FAILED: xpath=//result[@numFound=0]
[junit4] 2> xml response was: <?xml version="1.0" encoding="UTF-8"?>
[junit4] 2> <response>
[junit4] 2> <lst name="responseHeader"><int name="status">0</int><int name="QTime">2526</int></lst><result name="response" numFound="1" start="0"><doc><int name="id">529</int><arr name="range_facet_l"><long>529</long></arr><arr name="field_t"><str>what's inside?</str></arr><arr name="subject"><str>info</str></arr><long name="_version_">1476640896451608576</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2014-08-17T00:14:57.501Z</date></doc></result>
[junit4] 2> </response>
[junit4] 2>
[junit4] 2> request was:start=0&rows=20&version=2.2&qt=standard&q=id:529
[junit4] 2> 2015929 T5446 oasc.SolrException.log ERROR REQUEST FAILED: start=0&rows=20&version=2.2&qt=standard&q=id:529:java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=0]
[junit4] 2> xml response was: <?xml version="1.0" encoding="UTF-8"?>
[junit4] 2> <response>
[junit4] 2> <lst name="responseHeader"><int name="status">0</int><int name="QTime">2526</int></lst><result name="response" numFound="1" start="0"><doc><int name="id">529</int><arr name="range_facet_l"><long>529</long></arr><arr name="field_t"><str>what's inside?</str></arr><arr name="subject"><str>info</str></arr><long name="_version_">1476640896451608576</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2014-08-17T00:14:57.501Z</date></doc></result>
[junit4] 2> </response>
[junit4] 2>
[junit4] 2> request was:start=0&rows=20&version=2.2&qt=standard&q=id:529
[junit4] 2> at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:715)
[junit4] 2> at org.apache.solr.update.HardAutoCommitTest.testCommitWithin(HardAutoCommitTest.java:79)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:94)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:619)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
[junit4] 2> at java.lang.Thread.run(Thread.java:853)
[junit4] 2>
[junit4] 2> 2015930 T5446 oas.SolrTestCaseJ4.tearDown ###Ending testCommitWithin
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=HardAutoCommitTest -Dtests.method=testCommitWithin -Dtests.seed=9057D8056DEB331C -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ar_JO -Dtests.timezone=Israel -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 3.71s J0 | HardAutoCommitTest.testCommitWithin <<<
[junit4] > Throwable #1: java.lang.RuntimeException: Exception during query
[junit4] > at __randomizedtesting.SeedInfo.seed([9057D8056DEB331C:2A85B77DEEC5DD09]:0)
[junit4] > at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:722)
[junit4] > at org.apache.solr.update.HardAutoCommitTest.testCommitWithin(HardAutoCommitTest.java:79)
[junit4] > at java.lang.Thread.run(Thread.java:853)
[junit4] > Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=0]
[junit4] > xml response was: <?xml version="1.0" encoding="UTF-8"?>
[junit4] > <response>
[junit4] > <lst name="responseHeader"><int name="status">0</int><int name="QTime">2526</int></lst><result name="response" numFound="1" start="0"><doc><int name="id">529</int><arr name="range_facet_l"><long>529</long></arr><arr name="field_t"><str>what's inside?</str></arr><arr name="subject"><str>info</str></arr><long name="_version_">1476640896451608576</long><arr name="multiDefault"><str>muLti-Default</str></arr><int name="intDefault">42</int><date name="timestamp">2014-08-17T00:14:57.501Z</date></doc></result>
[junit4] > </response>
[junit4] > request was:start=0&rows=20&version=2.2&qt=standard&q=id:529
[junit4] > at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:715)
[junit4] > ... 41 more
[junit4] 2> 2015948 T5446 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> 2015949 T5446 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=-438346413
[junit4] 2> 2015949 T5446 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@ee2e94da
[junit4] 2> 2015962 T5446 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=1,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=1,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=151,transaction_logs_total_number=1}
[junit4] 2> 2015963 T5446 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 2015963 T5446 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 2015964 T5446 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 2015964 T5446 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 2015965 T5446 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 2015965 T5446 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.update.HardAutoCommitTest-9057D8056DEB331C-001/init-core-data-001 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.update.HardAutoCommitTest-9057D8056DEB331C-001/init-core-data-001;done=false>>]
[junit4] 2> 2015966 T5446 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.update.HardAutoCommitTest-9057D8056DEB331C-001/init-core-data-001
[junit4] 2> 2015966 T5446 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.update.HardAutoCommitTest-9057D8056DEB331C-001/init-core-data-001/index [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.update.HardAutoCommitTest-9057D8056DEB331C-001/init-core-data-001/index;done=false>>]
[junit4] 2> 2015966 T5446 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.update.HardAutoCommitTest-9057D8056DEB331C-001/init-core-data-001/index
[junit4] 2> NOTE: leaving temporary files on disk at: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.update.HardAutoCommitTest-9057D8056DEB331C-001
[junit4] 2> NOTE: test params are: codec=Lucene40, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=ar_JO, timezone=Israel
[junit4] 2> NOTE: Linux 3.13.0-32-generic x86/IBM Corporation 1.7.0 (32-bit)/cpus=8,threads=1,free=142282664,total=278790144
[junit4] 2> NOTE: All tests run in this JVM: [DeleteLastCustomShardedReplicaTest, StressHdfsTest, BlockDirectoryTest, TestComponentsName, DirectSolrSpellCheckerTest, DistributedExpandComponentTest, BasicFunctionalityTest, DirectUpdateHandlerTest, PluginInfoTest, SpellPossibilityIteratorTest, NoFacetTest, AnalyticsQueryTest, TestShardHandlerFactory, AtomicUpdatesTest, ZkNodePropsTest, TestFuzzyAnalyzedSuggestions, TestSolr4Spatial2, TestConfig, ZkControllerTest, ReplicationFactorTest, HdfsRecoveryZkTest, DistribCursorPagingTest, TestBlendedInfixSuggestions, DeleteInactiveReplicaTest, SolrCoreCheckLockOnStartupTest, StandardRequestHandlerTest, TestMiniSolrCloudCluster, HighlighterTest, TestCloudManagedSchema, TestSolrXmlPersistor, TestFaceting, OverseerTest, RecoveryZkTest, TestDocSet, StatelessScriptUpdateProcessorFactoryTest, TestSort, TestLMJelinekMercerSimilarityFactory, BasicDistributedZkTest, ShardRoutingTest, TestRandomMergePolicy, TestSurroundQueryParser, SignatureUpdateProcessorFactoryTest, ReturnFieldsTest, IndexSchemaTest, TestLRUCache, TestDistributedMissingSort, DOMUtilTest, XsltUpdateRequestHandlerTest, TestFastWriter, DistributedIntervalFacetingTest, QueryFacetTest, TestRemoteStreaming, TestFoldingMultitermQuery, TestDefaultSimilarityFactory, DocValuesMultiTest, DocExpirationUpdateProcessorFactoryTest, TestDocumentBuilder, TestRestManager, TestSolrJ, HighlighterConfigTest, EnumFieldTest, DeleteShardTest, TestDFRSimilarityFactory, TestRangeQuery, HdfsUnloadDistributedZkTest, TestReplicationHandler, SuggesterTest, TestComplexPhraseQParserPlugin, ExternalFileFieldSortTest, DisMaxRequestHandlerTest, TestDistributedSearch, TestStressVersions, TestStressReorder, RequiredFieldsTest, TestSchemaNameResource, SimpleFacetsTest, SSLMigrationTest, TermsComponentTest, DocValuesMissingTest, DocumentBuilderTest, TestModifyConfFiles, ResponseHeaderTest, TestExpandComponent, TestLMDirichletSimilarityFactory, PathHierarchyTokenizerFactoryTest, AsyncMigrateRouteKeyTest, TestSearchPerf, MinimalSchemaTest, TestAnalyzedSuggestions, AddBlockUpdateTest, TestFastLRUCache, TestShortCircuitedRequests, SolrCmdDistributorTest, UnloadDistributedZkTest, SuggesterTSTTest, OverseerStatusTest, TestMaxScoreQueryParser, TestSolrIndexConfig, CachingDirectoryFactoryTest, RollingRestartTest, TestReload, IndexBasedSpellCheckerTest, DistributedSpellCheckComponentTest, SolrIndexSplitterTest, TestMultiCoreConfBootstrap, HardAutoCommitTest]
[junit4] Completed on J0 in 4.94s, 1 test, 1 error <<< FAILURES!
[...truncated 620 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:474: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:496: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1350: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:965: There were test failures: 430 suites, 1782 tests, 1 error, 48 ignored (19 assumptions)
Total time: 107 minutes 21 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 32bit/ibm-j9-jdk7 -Xjit:exclude={org/apache/lucene/util/fst/FST.pack(IIF)Lorg/apache/lucene/util/fst/FST;}
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0_11) - Build #
11038 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/11038/
Java: 32bit/jdk1.8.0_11 -server -XX:+UseSerialGC
1 tests failed.
REGRESSION: org.apache.solr.cloud.RollingRestartTest.testDistribSearch
Error Message:
KeeperErrorCode = Session expired for /overseer_elect/leader
Stack Trace:
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
at __randomizedtesting.SeedInfo.seed([FB80BDFD38BA2245:7A6633E54FE54279]:0)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
at org.apache.solr.cloud.OverseerCollectionProcessor.getLeaderId(OverseerCollectionProcessor.java:487)
at org.apache.solr.cloud.OverseerCollectionProcessor.getLeaderNode(OverseerCollectionProcessor.java:478)
at org.apache.solr.cloud.RollingRestartTest.waitUntilOverseerDesignateIsLeader(RollingRestartTest.java:145)
at org.apache.solr.cloud.RollingRestartTest.restartWithRolesTest(RollingRestartTest.java:114)
at org.apache.solr.cloud.RollingRestartTest.doTest(RollingRestartTest.java:64)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869)
at sun.reflect.GeneratedMethodAccessor38.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11934 lines...]
[junit4] Suite: org.apache.solr.cloud.RollingRestartTest
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/init-core-data-001
[junit4] 2> 2574294 T7397 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (true)
[junit4] 2> 2574295 T7397 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4] 2> 2574298 T7397 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4] 2> 2574299 T7397 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2574300 T7398 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 2574400 T7397 oasc.ZkTestServer.run start zk server on port:44726
[junit4] 2> 2574401 T7397 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2574403 T7404 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a63876 name:ZooKeeperConnection Watcher:127.0.0.1:44726 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2574403 T7397 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2574403 T7397 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 2574406 T7397 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2574408 T7406 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1d6b21b name:ZooKeeperConnection Watcher:127.0.0.1:44726/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2574408 T7397 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2574408 T7397 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4] 2> 2574410 T7397 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4] 2> 2574411 T7397 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4] 2> 2574412 T7397 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4] 2> 2574413 T7397 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 2574414 T7397 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4] 2> 2574416 T7397 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 2574416 T7397 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4] 2> 2574418 T7397 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 2574418 T7397 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 2574420 T7397 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 2574420 T7397 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4] 2> 2574422 T7397 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 2574422 T7397 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4] 2> 2574423 T7397 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 2574424 T7397 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4] 2> 2574425 T7397 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 2574425 T7397 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
[junit4] 2> 2574427 T7397 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 2574427 T7397 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4] 2> 2574429 T7397 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 2574429 T7397 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 2574430 T7397 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 2574431 T7397 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4] 2> 2574432 T7397 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 2574433 T7397 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4] 2> 2574437 T7397 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2574438 T7408 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a2d7e8 name:ZooKeeperConnection Watcher:127.0.0.1:44726/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2574438 T7397 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2574764 T7397 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 2574767 T7397 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 2574802 T7397 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:59902
[junit4] 2> 2574803 T7397 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 2574804 T7397 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 2574804 T7397 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-002
[junit4] 2> 2574805 T7397 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-002/'
[junit4] 2> 2574818 T7397 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-002/solr.xml
[junit4] 2> 2574826 T7397 oasc.CoreContainer.<init> New CoreContainer 25896844
[junit4] 2> 2574827 T7397 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-002/]
[junit4] 2> 2574827 T7397 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 2574827 T7397 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 2574828 T7397 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 2574828 T7397 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 2574828 T7397 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 2574829 T7397 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 2574829 T7397 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 2574829 T7397 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 2574830 T7397 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 2574831 T7397 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 2574832 T7397 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 2574832 T7397 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 2574833 T7397 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 2574833 T7397 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:44726/solr
[junit4] 2> 2574833 T7397 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 2574835 T7397 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2574835 T7419 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1ef47b5 name:ZooKeeperConnection Watcher:127.0.0.1:44726 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2574836 T7397 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2574838 T7397 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2574838 T7421 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@de1ea7 name:ZooKeeperConnection Watcher:127.0.0.1:44726/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2574838 T7397 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2574840 T7397 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 2574842 T7397 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 2574845 T7397 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 2574847 T7397 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 2574849 T7397 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 2574850 T7397 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 2574851 T7397 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59902_
[junit4] 2> 2574852 T7397 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:59902_
[junit4] 2> 2574853 T7397 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 2574854 T7397 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 2574856 T7397 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 2574857 T7397 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:59902_
[junit4] 2> 2574858 T7397 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 2574859 T7397 oasc.Overseer.start Overseer (id=92289259838439428-127.0.0.1:59902_-n_0000000000) starting
[junit4] 2> 2574862 T7397 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 2574869 T7397 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 2574870 T7423 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 2574871 T7397 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 2574872 T7397 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 2574873 T7397 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2574875 T7422 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 2574878 T7425 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
[junit4] 2> 2574878 T7425 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 2574878 T7425 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 2574879 T7426 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2574880 T7422 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:59902",
[junit4] 2> "node_name":"127.0.0.1:59902_",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"control_collection",
[junit4] 2> "operation":"state"}
[junit4] 2> 2574880 T7422 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
[junit4] 2> 2574880 T7422 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 2574882 T7426 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2575879 T7425 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 2575879 T7425 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4] 2> 2575881 T7425 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 2575881 T7425 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 2575883 T7425 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 2575883 T7425 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-002/collection1/'
[junit4] 2> 2575884 T7425 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-002/collection1/lib/.svn/' to classloader
[junit4] 2> 2575884 T7425 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-002/collection1/lib/classes/' to classloader
[junit4] 2> 2575885 T7425 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-002/collection1/lib/README' to classloader
[junit4] 2> 2575907 T7425 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
[junit4] 2> 2575916 T7425 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 2575928 T7425 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 2575932 T7425 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 2576063 T7425 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 2576067 T7425 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 2576068 T7425 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 2576081 T7425 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2576083 T7425 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2576084 T7425 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 2576085 T7425 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 2576085 T7425 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 2576085 T7425 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 2576086 T7425 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 2576086 T7425 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 2576086 T7425 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
[junit4] 2> 2576087 T7425 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 2576087 T7425 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-002/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-001/control/data/
[junit4] 2> 2576087 T7425 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1e052e2
[junit4] 2> 2576087 T7425 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-001/control/data
[junit4] 2> 2576112 T7425 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-001/control/data/index/
[junit4] 2> 2576112 T7425 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-001/control/data/index' doesn't exist. Creating new index...
[junit4] 2> 2576112 T7425 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-001/control/data/index
[junit4] 2> 2576113 T7425 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=16, maxMergeAtOnceExplicit=35, maxMergedSegmentMB=92.9521484375, floorSegmentMB=1.7978515625, forceMergeDeletesPctAllowed=4.330957176408097, segmentsPerTier=32.0, maxCFSSegmentSizeMB=0.6572265625, noCFSRatio=0.0
[junit4] 2> 2585384 T7425 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-001/control/data/index,segFN=segments_1,generation=1}
[junit4] 2> 2585384 T7425 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 2585387 T7425 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 2585387 T7425 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 2585388 T7425 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 2585388 T7425 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 2585388 T7425 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 2585388 T7425 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 2585389 T7425 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 2585389 T7425 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 2585389 T7425 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 2585389 T7425 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 2585390 T7425 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 2585390 T7425 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 2585390 T7425 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 2585390 T7425 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 2585391 T7425 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 2585391 T7425 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 2585391 T7425 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 2585392 T7425 oasc.RequestHandlers.initHandlersFromConfig WARN Multiple requestHandler registered to the same name: /update ignoring: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 2585392 T7425 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 2585392 T7425 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 2585393 T7425 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 2585393 T7425 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 2585395 T7425 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2585395 T7425 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2585396 T7425 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2585397 T7425 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2585398 T7425 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2585399 T7425 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 2585399 T7425 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 2585399 T7425 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=38, maxMergeAtOnceExplicit=22, maxMergedSegmentMB=48.1552734375, floorSegmentMB=0.8603515625, forceMergeDeletesPctAllowed=26.22180068033101, segmentsPerTier=39.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 2585400 T7425 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-001/control/data/index,segFN=segments_1,generation=1}
[junit4] 2> 2585400 T7425 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 2585400 T7425 oass.SolrIndexSearcher.<init> Opening Searcher@1a1e12[collection1] main
[junit4] 2> 2585401 T7425 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
[junit4] 2> 2585402 T7425 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 2585402 T7425 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 2585402 T7425 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2585402 T7425 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 2585403 T7425 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2585403 T7425 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
[junit4] 2> 2585403 T7425 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2585404 T7425 oascc.SolrZkClient.makePath makePath: /configs/conf1/_rest_managed.json
[junit4] 2> 2585405 T7425 oasr.ManagedResourceStorage$ZooKeeperStorageIO$1.close Wrote 38 bytes to new znode /configs/conf1/_rest_managed.json
[junit4] 2> 2585405 T7425 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2585406 T7425 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 2585408 T7427 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1a1e12[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 2585409 T7425 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 2585409 T7430 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:59902 collection:control_collection shard:shard1
[junit4] 2> 2585410 T7397 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 2585410 T7397 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 2585411 T7430 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4] 2> 2585413 T7397 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2585414 T7432 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d72231 name:ZooKeeperConnection Watcher:127.0.0.1:44726/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2585414 T7397 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2585415 T7397 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2585415 T7430 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 2585416 T7397 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 2585417 T7430 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 2585417 T7430 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C3529 name=collection1 org.apache.solr.core.SolrCore@19b3001 url=https://127.0.0.1:59902/collection1 node=127.0.0.1:59902_ C3529_STATE=coll:control_collection core:collection1 props:{core=collection1, base_url=https://127.0.0.1:59902, node_name=127.0.0.1:59902_, state=down}
[junit4] 2> 2585417 T7430 C3529 P59902 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:59902/collection1/
[junit4] 2> 2585416 T7426 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2585417 T7430 C3529 P59902 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 2585418 T7430 C3529 P59902 oasc.SyncStrategy.syncToMe https://127.0.0.1:59902/collection1/ has no replicas
[junit4] 2> 2585418 T7430 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:59902/collection1/ shard1
[junit4] 2> 2585419 T7430 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4] 2> 2585423 T7426 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2585425 T7433 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2585427 T7426 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2585428 T7426 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2585428 T7433 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2585476 T7430 oasc.ZkController.register We are https://127.0.0.1:59902/collection1/ and leader is https://127.0.0.1:59902/collection1/
[junit4] 2> 2585476 T7430 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:59902
[junit4] 2> 2585476 T7430 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 2585476 T7430 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
[junit4] 2> 2585477 T7430 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 2585478 T7426 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2585479 T7422 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:59902",
[junit4] 2> "node_name":"127.0.0.1:59902_",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"control_collection",
[junit4] 2> "operation":"state"}
[junit4] 2> 2585580 T7426 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2585581 T7433 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 2585776 T7397 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
[junit4] 2> 2585777 T7397 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 2585780 T7397 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 2585781 T7397 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:50100
[junit4] 2> 2585783 T7397 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 2585783 T7397 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 2585784 T7397 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-003
[junit4] 2> 2585784 T7397 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-003/'
[junit4] 2> 2585801 T7397 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-003/solr.xml
[junit4] 2> 2585830 T7397 oasc.CoreContainer.<init> New CoreContainer 7117592
[junit4] 2> 2585831 T7397 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-003/]
[junit4] 2> 2585832 T7397 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 2585833 T7397 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 2585834 T7397 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 2585834 T7397 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 2585835 T7397 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 2585836 T7397 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 2585836 T7397 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 2585837 T7397 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 2585837 T7397 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 2585839 T7397 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 2585840 T7397 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 2585840 T7397 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 2585840 T7397 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 2585841 T7397 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:44726/solr
[junit4] 2> 2585841 T7397 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 2585842 T7397 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2585843 T7444 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1ca569e name:ZooKeeperConnection Watcher:127.0.0.1:44726 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2585843 T7397 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2585845 T7397 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2585845 T7446 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f05b9f name:ZooKeeperConnection Watcher:127.0.0.1:44726/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2585846 T7397 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2585849 T7397 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2586852 T7397 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50100_
[junit4] 2> 2586853 T7397 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50100_
[junit4] 2> 2586859 T7397 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 2586863 T7448 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 2586864 T7448 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 2586865 T7426 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2586865 T7448 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 2586867 T7422 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:50100",
[junit4] 2> "node_name":"127.0.0.1:50100_",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state"}
[junit4] 2> 2586868 T7422 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
[junit4] 2> 2586868 T7422 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 2586972 T7433 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2586972 T7426 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2586973 T7447 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2587866 T7448 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 2587866 T7448 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 2587867 T7448 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 2587867 T7448 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 2587868 T7448 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 2587868 T7448 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-003/collection1/'
[junit4] 2> 2587869 T7448 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-003/collection1/lib/.svn/' to classloader
[junit4] 2> 2587870 T7448 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-003/collection1/lib/classes/' to classloader
[junit4] 2> 2587870 T7448 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-003/collection1/lib/README' to classloader
[junit4] 2> 2587897 T7448 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
[junit4] 2> 2587909 T7448 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 2587910 T7448 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 2587916 T7448 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 2588040 T7448 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 2588045 T7448 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 2588047 T7448 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 2588055 T7448 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2588058 T7448 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2588060 T7448 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 2588061 T7448 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 2588061 T7448 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 2588061 T7448 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 2588062 T7448 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 2588062 T7448 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 2588063 T7448 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 2588063 T7448 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 2588063 T7448 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-003/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-001/jetty1/
[junit4] 2> 2588064 T7448 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1e052e2
[junit4] 2> 2588064 T7448 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-001/jetty1
[junit4] 2> 2588065 T7448 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-001/jetty1/index/
[junit4] 2> 2588065 T7448 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-001/jetty1/index' doesn't exist. Creating new index...
[junit4] 2> 2588065 T7448 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-001/jetty1/index
[junit4] 2> 2588066 T7448 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=16, maxMergeAtOnceExplicit=35, maxMergedSegmentMB=92.9521484375, floorSegmentMB=1.7978515625, forceMergeDeletesPctAllowed=4.330957176408097, segmentsPerTier=32.0, maxCFSSegmentSizeMB=0.6572265625, noCFSRatio=0.0
[junit4] 2> 2588073 T7448 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-001/jetty1/index,segFN=segments_1,generation=1}
[junit4] 2> 2588074 T7448 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 2588076 T7448 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 2588077 T7448 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 2588077 T7448 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 2588077 T7448 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 2588077 T7448 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 2588077 T7448 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 2588078 T7448 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 2588078 T7448 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 2588078 T7448 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 2588078 T7448 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 2588079 T7448 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 2588079 T7448 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 2588079 T7448 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 2588080 T7448 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 2588080 T7448 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 2588080 T7448 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 2588080 T7448 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 2588081 T7448 oasc.RequestHandlers.initHandlersFromConfig WARN Multiple requestHandler registered to the same name: /update ignoring: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 2588081 T7448 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 2588081 T7448 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 2588081 T7448 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 2588082 T7448 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 2588084 T7448 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2588084 T7448 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2588085 T7448 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2588086 T7448 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2588086 T7448 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2588087 T7448 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 2588088 T7448 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 2588088 T7448 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=38, maxMergeAtOnceExplicit=22, maxMergedSegmentMB=48.1552734375, floorSegmentMB=0.8603515625, forceMergeDeletesPctAllowed=26.22180068033101, segmentsPerTier=39.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 2588089 T7448 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-001/jetty1/index,segFN=segments_1,generation=1}
[junit4] 2> 2588089 T7448 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 2588089 T7448 oass.SolrIndexSearcher.<init> Opening Searcher@1addbd8[collection1] main
[junit4] 2> 2588089 T7448 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 2588090 T7448 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 2588091 T7448 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 2588091 T7448 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2588091 T7448 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 2588092 T7448 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2588092 T7448 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json
[junit4] 2> 2588093 T7448 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2588093 T7448 oasr.ManagedResource.reloadFromStorage Loaded initArgs {} for /rest/managed
[junit4] 2> 2588093 T7448 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 2588095 T7449 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1addbd8[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 2588097 T7448 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 2588097 T7452 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:50100 collection:collection1 shard:shard2
[junit4] 2> 2588098 T7397 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 2588098 T7452 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 2588098 T7397 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 2588101 T7452 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4] 2> 2588102 T7426 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2588102 T7452 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 2588102 T7452 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C3530 name=collection1 org.apache.solr.core.SolrCore@10dfc4d url=https://127.0.0.1:50100/collection1 node=127.0.0.1:50100_ C3530_STATE=coll:collection1 core:collection1 props:{core=collection1, base_url=https://127.0.0.1:50100, node_name=127.0.0.1:50100_, state=down}
[junit4] 2> 2588103 T7452 C3530 P50100 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:50100/collection1/
[junit4] 2> 2588103 T7452 C3530 P50100 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 2588103 T7452 C3530 P50100 oasc.SyncStrategy.syncToMe https://127.0.0.1:50100/collection1/ has no replicas
[junit4] 2> 2588103 T7452 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:50100/collection1/ shard2
[junit4] 2> 2588103 T7452 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 2588106 T7426 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2588209 T7426 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2588209 T7447 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2588209 T7433 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2588257 T7452 oasc.ZkController.register We are https://127.0.0.1:50100/collection1/ and leader is https://127.0.0.1:50100/collection1/
[junit4] 2> 2588257 T7452 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:50100
[junit4] 2> 2588257 T7452 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 2588257 T7452 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 2588258 T7452 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 2588259 T7426 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2588260 T7422 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:50100",
[junit4] 2> "node_name":"127.0.0.1:50100_",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state"}
[junit4] 2> 2588362 T7426 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2588362 T7433 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2588363 T7447 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4] 2> 2588561 T7397 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
[junit4] 2> 2588562 T7397 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4] 2> 2588565 T7397 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
[junit4] 2> 2588566 T7397 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:50408
[junit4] 2> 2588567 T7397 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4] 2> 2588568 T7397 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4] 2> 2588568 T7397 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-004
[junit4] 2> 2588569 T7397 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-004/'
[junit4] 2> 2588582 T7397 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-004/solr.xml
[junit4] 2> 2588590 T7397 oasc.CoreContainer.<init> New CoreContainer 9393942
[junit4] 2> 2588591 T7397 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-004/]
[junit4] 2> 2588592 T7397 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
[junit4] 2> 2588592 T7397 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to:
[junit4] 2> 2588593 T7397 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4] 2> 2588593 T7397 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4] 2> 2588593 T7397 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4] 2> 2588594 T7397 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4] 2> 2588594 T7397 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4] 2> 2588595 T7397 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4] 2> 2588595 T7397 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4] 2> 2588596 T7397 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
[junit4] 2> 2588597 T7397 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
[junit4] 2> 2588598 T7397 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
[junit4] 2> 2588598 T7397 oasc.CoreContainer.load Host Name: 127.0.0.1
[junit4] 2> 2588598 T7397 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:44726/solr
[junit4] 2> 2588599 T7397 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4] 2> 2588599 T7397 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2588600 T7463 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3e63fb name:ZooKeeperConnection Watcher:127.0.0.1:44726 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2588601 T7397 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2588603 T7397 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 2588604 T7465 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@8fb60 name:ZooKeeperConnection Watcher:127.0.0.1:44726/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 2588604 T7397 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 2588611 T7397 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 2589613 T7397 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50408_
[junit4] 2> 2589615 T7397 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50408_
[junit4] 2> 2589620 T7397 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 2589625 T7467 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
[junit4] 2> 2589625 T7467 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 2589627 T7426 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2589627 T7467 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4] 2> 2589629 T7422 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:50408",
[junit4] 2> "node_name":"127.0.0.1:50408_",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":null,
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state"}
[junit4] 2> 2589630 T7422 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4] 2> 2589630 T7422 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 2589734 T7426 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2589734 T7466 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2589734 T7447 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2589734 T7433 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2590627 T7467 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4] 2> 2590628 T7467 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4] 2> 2590629 T7467 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4] 2> 2590629 T7467 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 2590631 T7467 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 2590631 T7467 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-004/collection1/'
[junit4] 2> 2590632 T7467 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-004/collection1/lib/.svn/' to classloader
[junit4] 2> 2590632 T7467 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-004/collection1/lib/classes/' to classloader
[junit4] 2> 2590633 T7467 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-004/collection1/lib/README' to classloader
[junit4] 2> 2590653 T7467 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
[junit4] 2> 2590662 T7467 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4] 2> 2590664 T7467 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4] 2> 2590668 T7467 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4] 2> 2590790 T7467 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4] 2> 2590793 T7467 oass.IndexSchema.readSchema default search field in schema is text
[junit4] 2> 2590794 T7467 oass.IndexSchema.readSchema unique key field: id
[junit4] 2> 2590806 T7467 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2590808 T7467 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4] 2> 2590810 T7467 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 2590811 T7467 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 2590811 T7467 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 2590812 T7467 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4] 2> 2590813 T7467 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4] 2> 2590813 T7467 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4] 2> 2590813 T7467 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
[junit4] 2> 2590814 T7467 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4] 2> 2590814 T7467 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-004/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-001/jetty2/
[junit4] 2> 2590814 T7467 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1e052e2
[junit4] 2> 2590814 T7467 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-001/jetty2
[junit4] 2> 2590815 T7467 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-001/jetty2/index/
[junit4] 2> 2590815 T7467 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-001/jetty2/index' doesn't exist. Creating new index...
[junit4] 2> 2590816 T7467 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-001/jetty2/index
[junit4] 2> 2590816 T7467 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=16, maxMergeAtOnceExplicit=35, maxMergedSegmentMB=92.9521484375, floorSegmentMB=1.7978515625, forceMergeDeletesPctAllowed=4.330957176408097, segmentsPerTier=32.0, maxCFSSegmentSizeMB=0.6572265625, noCFSRatio=0.0
[junit4] 2> 2590825 T7467 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-001/jetty2/index,segFN=segments_1,generation=1}
[junit4] 2> 2590825 T7467 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 2590830 T7467 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
[junit4] 2> 2590830 T7467 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
[junit4] 2> 2590831 T7467 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4] 2> 2590831 T7467 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
[junit4] 2> 2590831 T7467 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4] 2> 2590832 T7467 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
[junit4] 2> 2590832 T7467 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 2590833 T7467 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4] 2> 2590833 T7467 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
[junit4] 2> 2590834 T7467 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 2590834 T7467 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 2590835 T7467 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 2590835 T7467 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 2590836 T7467 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4] 2> 2590836 T7467 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4] 2> 2590836 T7467 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4] 2> 2590837 T7467 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4] 2> 2590837 T7467 oasc.RequestHandlers.initHandlersFromConfig WARN Multiple requestHandler registered to the same name: /update ignoring: org.apache.solr.handler.UpdateRequestHandler
[junit4] 2> 2590837 T7467 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4] 2> 2590838 T7467 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4] 2> 2590838 T7467 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
[junit4] 2> 2590838 T7467 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
[junit4] 2> 2590841 T7467 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2590842 T7467 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2590842 T7467 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2590843 T7467 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2590844 T7467 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4] 2> 2590845 T7467 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 2590846 T7467 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 2590846 T7467 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=38, maxMergeAtOnceExplicit=22, maxMergedSegmentMB=48.1552734375, floorSegmentMB=0.8603515625, forceMergeDeletesPctAllowed=26.22180068033101, segmentsPerTier=39.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 2590847 T7467 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4] 2> commit{dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001/tempDir-001/jetty2/index,segFN=segments_1,generation=1}
[junit4] 2> 2590848 T7467 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4] 2> 2590848 T7467 oass.SolrIndexSearcher.<init> Opening Searcher@1acc8a8[collection1] main
[junit4] 2> 2590848 T7467 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
[junit4] 2> 2590849 T7467 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
[junit4] 2> 2590849 T7467 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
[junit4] 2> 2590850 T7467 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2590850 T7467 oasr.RestManager.init Initializing RestManager with initArgs: {}
[junit4] 2> 2590850 T7467 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2590852 T7467 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream Read 38 bytes from znode /configs/conf1/_rest_managed.json
[junit4] 2> 2590852 T7467 oasr.ManagedResourceStorage.load Loaded LinkedHashMap at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2590852 T7467 oasr.ManagedResource.reloadFromStorage Loaded initArgs {} for /rest/managed
[junit4] 2> 2590853 T7467 oasr.RestManager.init Initializing 0 registered ManagedResources
[junit4] 2> 2590856 T7468 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1acc8a8[collection1] main{UninvertingDirectoryReader()}
[junit4] 2> 2590858 T7467 oasc.CoreContainer.registerCore registering core: collection1
[junit4] 2> 2590858 T7471 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:50408 collection:collection1 shard:shard1
[junit4] 2> 2590859 T7397 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4] 2> 2590859 T7471 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 2590860 T7397 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4] 2> 2590863 T7397 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):330
[junit4] 2> 2590864 T7471 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4] 2> 2590865 T7471 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4] 2> 2590866 T7471 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4] 2> ASYNC NEW_CORE C3531 name=collection1 org.apache.solr.core.SolrCore@d2e3d8 url=https://127.0.0.1:50408/collection1 node=127.0.0.1:50408_ C3531_STATE=coll:collection1 core:collection1 props:{core=collection1, base_url=https://127.0.0.1:50408, node_name=127.0.0.1:50408_, state=down}
[junit4] 2> 2590866 T7471 C3531 P50408 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:50408/collection1/
[junit4] 2> 2590866 T7426 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2590866 T7471 C3531 P50408 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4] 2> 2590866 T7471 C3531 P50408 oasc.SyncStrategy.syncToMe https://127.0.0.1:50408/collection1/ has no replicas
[junit4] 2> 2590867 T7471 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:50408/collection1/ shard1
[junit4] 2> 2590867 T7471 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 2590870 T7426 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2590973 T7426 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2590973 T7433 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2590973 T7447 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2590973 T7466 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2591021 T7471 oasc.ZkController.register We are https://127.0.0.1:50408/collection1/ and leader is https://127.0.0.1:50408/collection1/
[junit4] 2> 2591021 T7471 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:50408
[junit4] 2> 2591021 T7471 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4] 2> 2591021 T7471 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
[junit4] 2> 2591022 T7471 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4] 2> 2591023 T7426 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 2591026 T7422 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4] 2> "core":"collection1",
[junit4] 2> "core_node_name":"core_node2",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:50408",
[junit4] 2> "node_name":"127.0.0.1:50408_",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "operation":"state"}
[junit4] 2> 2591129 T7466 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2591130 T7426 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2591130 T7447 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2591130 T7433 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4] 2> 2591865 T7397 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4] 2> 2591867 T7397 oasc.RollingRestartTest.restartWithRolesTest Current overseer leader = 127.0.0.1:59902_
[junit4] 1> / (8)
[junit4] 1> /configs (1)
[junit4] 1> /configs/conf1 (12)
[junit4] 1> /configs/conf1/currency.xml (0)
[junit4] 1> DATA: ...supressed...
[junit4] 1> /configs/conf1/enumsConfig.xml (0)
[junit4] 1> DATA: ...supressed...
[junit4] 1> /configs/conf1/protwords.txt (0)
[junit4] 1> DATA: ...supressed...
[junit4] 1> /configs/conf1/old_synonyms.txt (0)
[junit4] 1> DATA: ...supressed...
[junit4] 1> /configs/conf1/solrconfig.xml (0)
[junit4] 1> DATA: ...supressed...
[junit4] 1> /configs/conf1/synonyms.txt (0)
[junit4] 1> DATA: ...supressed...
[junit4] 1> /configs/conf1/stopwords.txt (0)
[junit4] 1> DATA: ...supressed...
[junit4] 1> /configs/conf1/schema.xml (0)
[junit4] 1> DATA: ...supressed...
[junit4] 1> /configs/conf1/solrconfig.snippet.randomindexconfig.xml (0)
[junit4] 1> DATA: ...supressed...
[junit4] 1> /configs/conf1/mapping-ISOLatin1Accent.txt (0)
[junit4] 1> DATA: ...supressed...
[junit4] 1> /configs/conf1/_rest_managed.json (0)
[junit4] 1> DATA:
[junit4] 1> {
[junit4] 1> "initArgs":{},
[junit4] 1> "managedList":[]}
[junit4] 1> /configs/conf1/open-exchange-rates.json (0)
[junit4] 1> DATA:
[junit4] 1> {
[junit4] 1> "disclaimer": "This data is not real, it was synthetically created to match currency.xml. It is modeled after the data format available from openexchangerates.org. See https://openexchangerates.org/documentation for details",
[junit4] 1> "license": "http://www.apache.org/licenses/LICENSE-2.0",
[junit4] 1> "timestamp": 1332070464,
[junit4] 1>
[junit4] 1>
[junit4] 1> "IMPORTANT NOTE": "In order for tests to work, this data must be kept in sync with ./currency.xml",
[junit4] 1>
[junit4] 1>
[junit4] 1> "base": "USD",
[junit4] 1> "rates": {
[junit4] 1> "USD": 1,
[junit4] 1> "JPY": 81.29,
[junit4] 1> "EUR": 2.5,
[junit4] 1> "GBP": 0.5,
[junit4] 1> "MXN": 2.0
[junit4] 1> }
[junit4] 1> }
[junit4] 1>
[junit4] 1> /overseer (6)
[junit4] 1> DATA:
[junit4] 1>
[junit4] 1> /overseer/collection-queue-work (0)
[junit4] 1> /overseer/queue-work (0)
[junit4] 1> /overseer/collection-map-failure (0)
[junit4] 1> /overseer/collection-map-completed (0)
[junit4] 1> /overseer/queue (0)
[junit4] 1> /overseer/collection-map-running (0)
[junit4] 1> /aliases.json (0)
[junit4] 1> /collections (2)
[junit4] 1> /collections/collection1 (3)
[junit4] 1> DATA:
[junit4] 1> {"configName":"conf1"}
[junit4] 1> /collections/collection1/shards (0)
[junit4] 1> /collections/collection1/leaders (2)
[junit4] 1> /collections/collection1/leaders/shard2 (0)
[junit4] 1> DATA:
[junit4] 1> {
[junit4] 1> "core":"collection1",
[junit4] 1> "base_url":"https://127.0.0.1:50100",
[junit4] 1> "node_name":"127.0.0.1:50100_"}
[junit4] 1> /collections/collection1/leaders/shard1 (0)
[junit4] 1> DATA:
[junit4] 1> {
[junit4] 1> "core":"collection1",
[...truncated too long message...]
7034 T7542 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:50408_
[junit4] 2> 2658534 T7577 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:321)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:89)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:293)
[junit4] 2>
[junit4] 2> 2658535 T7571 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4] 2> 2658559 T7397 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
[junit4] 2> 2663549 T7397 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4] 2> 2663550 T7397 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:44726 44726
[junit4] 2> 2664255 T7398 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:44726 44726
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=RollingRestartTest -Dtests.method=testDistribSearch -Dtests.seed=FB80BDFD38BA2245 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=hr -Dtests.timezone=America/Montevideo -Dtests.file.encoding=UTF-8
[junit4] ERROR 90.1s J0 | RollingRestartTest.testDistribSearch <<<
[junit4] > Throwable #1: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] > at __randomizedtesting.SeedInfo.seed([FB80BDFD38BA2245:7A6633E54FE54279]:0)
[junit4] > at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] > at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] > at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
[junit4] > at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
[junit4] > at org.apache.solr.cloud.OverseerCollectionProcessor.getLeaderId(OverseerCollectionProcessor.java:487)
[junit4] > at org.apache.solr.cloud.OverseerCollectionProcessor.getLeaderNode(OverseerCollectionProcessor.java:478)
[junit4] > at org.apache.solr.cloud.RollingRestartTest.waitUntilOverseerDesignateIsLeader(RollingRestartTest.java:145)
[junit4] > at org.apache.solr.cloud.RollingRestartTest.restartWithRolesTest(RollingRestartTest.java:114)
[junit4] > at org.apache.solr.cloud.RollingRestartTest.doTest(RollingRestartTest.java:64)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 2664378 T7397 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.RollingRestartTest-FB80BDFD38BA2245-001
[junit4] 2> NOTE: test params are: codec=Lucene42, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=hr, timezone=America/Montevideo
[junit4] 2> NOTE: Linux 3.13.0-32-generic i386/Oracle Corporation 1.8.0_11 (32-bit)/cpus=8,threads=1,free=220192816,total=470396928
[junit4] 2> NOTE: All tests run in this JVM: [TestManagedResource, ConnectionManagerTest, TestLuceneMatchVersion, InfoHandlerTest, FieldMutatingUpdateProcessorTest, TestCloudManagedSchemaConcurrent, TestDocSet, TestReload, TestSolr4Spatial, ExternalFileFieldSortTest, QueryResultKeyTest, TestSolrQueryParserResource, TestFieldSortValues, EchoParamsTest, BlockCacheTest, QueryFacetTest, TestSerializedLuceneMatchVersion, ResponseHeaderTest, TestDefaultSimilarityFactory, TestCopyFieldCollectionResource, DirectSolrSpellCheckerTest, MergeStrategyTest, TestBlendedInfixSuggestions, AbstractAnalyticsFacetTest, TestManagedStopFilterFactory, DistributedFacetPivotLongTailTest, TestStressRecovery, TestSolrCoreProperties, TestFuzzyAnalyzedSuggestions, CoreAdminCreateDiscoverTest, DistributedQueryComponentCustomSortTest, TestDocumentBuilder, SyncSliceTest, TestIntervalFaceting, ClusterStateUpdateTest, DeleteShardTest, TestRandomMergePolicy, OverseerRolesTest, BasicDistributedZkTest, SignatureUpdateProcessorFactoryTest, TestFoldingMultitermQuery, TestInitQParser, PathHierarchyTokenizerFactoryTest, FileUtilsTest, TestDistributedSearch, TestSurroundQueryParser, TestFaceting, TestSolrDeletionPolicy2, TestReloadAndDeleteDocs, UnloadDistributedZkTest, SuggesterTSTTest, HdfsRecoveryZkTest, HttpPartitionTest, TestCodecSupport, DocumentBuilderTest, BadComponentTest, SolrIndexSplitterTest, PreAnalyzedUpdateProcessorTest, TestQuerySenderListener, DistributedFacetPivotLargeTest, QueryEqualityTest, SpellingQueryConverterTest, TestFieldTypeResource, TestCollapseQParserPlugin, TestBinaryResponseWriter, SpellPossibilityIteratorTest, TestLMJelinekMercerSimilarityFactory, OpenExchangeRatesOrgProviderTest, TestStressReorder, TestSort, ShardSplitTest, TestSolrXml, TestArbitraryIndexDir, SharedFSAutoReplicaFailoverUtilsTest, TestNonDefinedSimilarityFactory, TestFastLRUCache, CustomCollectionTest, XsltUpdateRequestHandlerTest, ChaosMonkeyNothingIsSafeTest, TestSolrDeletionPolicy1, TestReRankQParserPlugin, DistributedFacetPivotSmallTest, TestReplicationHandlerBackup, SystemInfoHandlerTest, TestHashPartitioner, PluginInfoTest, TestSchemaResource, TestLRUCache, TestShardHandlerFactory, TestTrie, TestSuggestSpellingConverter, TestSchemaVersionResource, MBeansHandlerTest, FastVectorHighlighterTest, TestSolrJ, TestOverriddenPrefixQueryForCustomFieldType, MoreLikeThisHandlerTest, TestRemoteStreaming, JSONWriterTest, TestRTGBase, ZkControllerTest, DistribCursorPagingTest, TestFastWriter, TestConfigSets, OverseerTest, ShardRoutingTest, BasicZkTest, RecoveryZkTest, FullSolrCloudDistribCmdsTest, TestRandomFaceting, LeaderElectionTest, ZkSolrClientTest, TestZkChroot, TestRandomDVFaceting, DistributedSpellCheckComponentTest, TestRangeQuery, SimpleFacetsTest, SolrCoreTest, SpellCheckComponentTest, SolrCmdDistributorTest, BadIndexSchemaTest, BasicFunctionalityTest, DirectUpdateHandlerTest, OverseerCollectionProcessorTest, TestIndexSearcher, ShowFileRequestHandlerTest, SuggesterFSTTest, CoreAdminHandlerTest, WordBreakSolrSpellCheckerTest, TestPseudoReturnFields, TestUpdate, StatelessScriptUpdateProcessorFactoryTest, StandardRequestHandlerTest, DocumentAnalysisRequestHandlerTest, TestQueryTypes, PrimitiveFieldTypeTest, FileBasedSpellCheckerTest, XmlUpdateRequestHandlerTest, RequiredFieldsTest, LoggingHandlerTest, IndexSchemaTest, ReturnFieldsTest, UpdateRequestProcessorFactoryTest, TestAnalyzedSuggestions, JsonLoaderTest, TestPartialUpdateDeduplication, TestPHPSerializedResponseWriter, CSVRequestHandlerTest, SearchHandlerTest, HighlighterConfigTest, UpdateParamsTest, ResponseLogComponentTest, MinimalSchemaTest, TestConfig, SynonymTokenizerTest, TestDFRSimilarityFactory, TestLMDirichletSimilarityFactory, TestBM25SimilarityFactory, TestJmxMonitoredMap, DateMathParserTest, TestSystemIdResolver, DOMUtilTest, ClusterStateTest, ZkNodePropsTest, CircularListTest, SolrTestCaseJ4Test, TestCursorMarkWithoutUniqueKey, TestDistributedMissingSort, TestHighlightDedupGrouping, TestTolerantSearch, AbstractAnalyticsStatsTest, NoFacetTest, FunctionTest, AsyncMigrateRouteKeyTest, MigrateRouteKeyTest, ReplicationFactorTest, RollingRestartTest]
[junit4] Completed on J0 in 90.12s, 1 test, 1 error <<< FAILURES!
[...truncated 228 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:474: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:454: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:496: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1350: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:965: There were test failures: 430 suites, 1782 tests, 1 error, 40 ignored (19 assumptions)
Total time: 99 minutes 28 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 32bit/jdk1.8.0_11 -server -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Re: [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.7.0_65) - Build
# 11037 - Still Failing!
Posted by Chris Hostetter <ho...@fucit.org>.
https://issues.apache.org/jira/browse/SOLR-6386
: Date: Sat, 16 Aug 2014 17:25:45 +0000 (UTC)
: From: Policeman Jenkins Server <je...@thetaphi.de>
: Reply-To: dev@lucene.apache.org
: To: dev@lucene.apache.org
: Subject: [JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.7.0_65) - Build # 11037
: - Still Failing!
:
: Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/11037/
: Java: 32bit/jdk1.7.0_65 -server -XX:+UseConcMarkSweepGC
:
: 2 tests failed.
: FAILED: org.apache.solr.TestDistributedSearch.testDistribSearch
:
: Error Message:
: .facet_counts.facet_fields.a_n_tdt.1970-01-01T00:00:00Z!=2010-04-20T11:00:00Z (unordered or missing)
:
: Stack Trace:
: junit.framework.AssertionFailedError: .facet_counts.facet_fields.a_n_tdt.1970-01-01T00:00:00Z!=2010-04-20T11:00:00Z (unordered or missing)
: at __randomizedtesting.SeedInfo.seed([5547E29057A024AB:D4A16C8820FF4497]:0)
: at junit.framework.Assert.fail(Assert.java:50)
: at org.apache.solr.BaseDistributedSearchTestCase.compareSolrResponses(BaseDistributedSearchTestCase.java:842)
: at org.apache.solr.BaseDistributedSearchTestCase.compareResponses(BaseDistributedSearchTestCase.java:861)
: at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:562)
: at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:542)
: at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:521)
: at org.apache.solr.TestDistributedSearch.doTest(TestDistributedSearch.java:180)
: at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:875)
: at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
: at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
: at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
: at java.lang.reflect.Method.invoke(Method.java:606)
: at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
: at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
: at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
: at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
: at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
: at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
: at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
: at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
: at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
: at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
: at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
: at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
: at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
: at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
: at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
: at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
: at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
: at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
: at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
: at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
: at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
: at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
: at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
: at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
: at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
: at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
: at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
: at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
: at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
: at java.lang.Thread.run(Thread.java:745)
:
:
: FAILED: org.apache.solr.cloud.OverseerTest.testOverseerFailure
:
: Error Message:
: KeeperErrorCode = NoNode for /collections/collection1/leader_elect/shard1/election
:
: Stack Trace:
: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/collection1/leader_elect/shard1/election
: at __randomizedtesting.SeedInfo.seed([5547E29057A024AB:514F6D634505CB8A]:0)
: at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
: at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
: at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
: at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:290)
: at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:287)
: at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
: at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:287)
: at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
: at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:155)
: at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:314)
: at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
: at org.apache.solr.cloud.OverseerTest$MockZKController.publishState(OverseerTest.java:157)
: at org.apache.solr.cloud.OverseerTest.testOverseerFailure(OverseerTest.java:662)
: at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
: at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
: at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
: at java.lang.reflect.Method.invoke(Method.java:606)
: at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
: at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
: at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
: at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
: at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
: at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
: at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
: at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
: at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
: at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
: at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
: at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
: at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
: at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
: at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
: at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
: at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
: at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
: at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
: at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
: at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
: at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
: at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
: at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
: at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
: at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
: at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
: at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
: at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
: at java.lang.Thread.run(Thread.java:745)
:
:
:
:
: Build Log:
: [...truncated 11366 lines...]
: [junit4] Suite: org.apache.solr.cloud.OverseerTest
: [junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerTest-5547E29057A024AB-001/init-core-data-001
: [junit4] 2> 1604704 T4268 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (false)
: [junit4] 2> 1604704 T4268 oas.SolrTestCaseJ4.initCore ####initCore
: [junit4] 2> 1604704 T4268 oas.SolrTestCaseJ4.initCore ####initCore end
: [junit4] 2> 1604707 T4268 oas.SolrTestCaseJ4.setUp ###Starting testPlaceholders
: [junit4] 2> 1604708 T4268 oasc.ZkTestServer.run STARTING ZK TEST SERVER
: [junit4] 1> client port:0.0.0.0/0.0.0.0:0
: [junit4] 2> 1604708 T4269 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
: [junit4] 2> 1604816 T4268 oasc.ZkTestServer.run start zk server on port:46180
: [junit4] 2> 1604817 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
: [junit4] 2> 1604820 T4275 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@cacff9 name:ZooKeeperConnection Watcher:127.0.0.1:46180/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 1604820 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
: [junit4] 2> 1604821 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
: [junit4] 2> 1604822 T4277 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a5eef5 name:ZooKeeperConnection Watcher:127.0.0.1:46180 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 1604822 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
: [junit4] 2> 1604824 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
: [junit4] 2> 1604825 T4279 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@9022ad name:ZooKeeperConnection Watcher:127.0.0.1:46180 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 1604825 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
: [junit4] 2> 1604825 T4268 oascc.SolrZkClient.makePath makePath: /solr
: [junit4] 2> 1604828 T4268 oascc.SolrZkClient.makePath makePath: /live_nodes
: [junit4] 2> 1604830 T4268 oascc.SolrZkClient.makePath makePath: /clusterstate.json
: [junit4] 2> 1604831 T4268 oascc.SolrZkClient.makePath makePath: /aliases.json
: [junit4] 2> 1604832 T4268 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
: [junit4] 2> 1604839 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
: [junit4] 2> 1604840 T4281 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ada0d1 name:ZooKeeperConnection Watcher:127.0.0.1:46180/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 1604840 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
: [junit4] 2> 1604841 T4268 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
: [junit4] 2> 1604842 T4268 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
: [junit4] 2> 1604851 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
: [junit4] 2> 1604852 T4285 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e7e1e7 name:ZooKeeperConnection Watcher:127.0.0.1:46180/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 1604852 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
: [junit4] 2> 1604853 T4268 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: retry=false
: [junit4] 2> 1604854 T4268 oascc.SolrZkClient.makePath makePath: /overseer_elect
: [junit4] 2> 1604855 T4268 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
: [junit4] 2> 1604857 T4268 oasc.Overseer.close Overseer (id=null) closing
: [junit4] 2> 1604858 T4268 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:46180_solr
: [junit4] 2> 1604858 T4268 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
: [junit4] 2> 1604860 T4268 oasc.Overseer.start Overseer (id=92288351107088388-127.0.0.1:46180_solr-n_0000000000) starting
: [junit4] 2> 1604861 T4268 oascc.SolrZkClient.makePath makePath: /overseer/queue
: [junit4] 2> 1604862 T4268 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
: [junit4] 2> 1604864 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
: [junit4] 2> 1604866 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
: [junit4] 2> 1604868 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
: [junit4] 2> 1604870 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
: [junit4] 2> 1604873 T4268 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
: [junit4] 2> 1604873 T4287 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
: [junit4] 2> 1604874 T4286 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
: [junit4] 2> 1604876 T4286 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=12 message={
: [junit4] 2> "operation":"state",
: [junit4] 2> "state":"recovering",
: [junit4] 2> "node_name":"node1",
: [junit4] 2> "core":"core1",
: [junit4] 2> "core_node_name":"node1",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "numShards":"12",
: [junit4] 2> "base_url":"http://node1/solr/"}
: [junit4] 2> 1604876 T4286 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2, shard3, shard4, shard5, shard6, shard7, shard8, shard9, shard10, shard11, shard12]
: [junit4] 2> 1604877 T4286 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
: [junit4] 2> 1604878 T4289 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
: [junit4] 2> 1604879 T4282 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 1604879 T4283 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 1605376 T4268 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
: [junit4] 2> 1605381 T4268 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
: [junit4] 2> 1605385 T4289 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
: [junit4] 2> 1605389 T4268 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:46180 46180
: [junit4] 2> 1605947 T4269 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:46180 46180
: [junit4] 2> 1605949 T4268 oas.SolrTestCaseJ4.tearDown ###Ending testPlaceholders
: [junit4] 2> 1605949 T4268 oasc.Overseer.close Overseer (id=92288351107088388-127.0.0.1:46180_solr-n_0000000000) closing
: [junit4] 2> 1605950 T4286 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:46180_solr
: [junit4] 2> 1605951 T4290 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
: [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
: [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
: [junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
: [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
: [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:321)
: [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:89)
: [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:293)
: [junit4] 2>
: [junit4] 2> 1605955 T4268 oas.SolrTestCaseJ4.setUp ###Starting testShardAssignment
: [junit4] 2> 1605955 T4268 oasc.ZkTestServer.run STARTING ZK TEST SERVER
: [junit4] 1> client port:0.0.0.0/0.0.0.0:0
: [junit4] 2> 1605956 T4291 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
: [junit4] 2> 1606056 T4268 oasc.ZkTestServer.run start zk server on port:51454
: [junit4] 2> 1606057 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
: [junit4] 2> 1606059 T4297 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5c931c name:ZooKeeperConnection Watcher:127.0.0.1:51454 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 1606059 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
: [junit4] 2> 1606061 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
: [junit4] 2> 1606062 T4299 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@cca746 name:ZooKeeperConnection Watcher:127.0.0.1:51454 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 1606062 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
: [junit4] 2> 1606063 T4268 oascc.SolrZkClient.makePath makePath: /solr
: [junit4] 2> 1606065 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
: [junit4] 2> 1606066 T4301 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@315e42 name:ZooKeeperConnection Watcher:127.0.0.1:51454/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 1606066 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
: [junit4] 2> 1606067 T4268 oascc.SolrZkClient.makePath makePath: /live_nodes
: [junit4] 2> 1606068 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
: [junit4] 2> 1606069 T4303 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@775b26 name:ZooKeeperConnection Watcher:127.0.0.1:51454/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 1606069 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
: [junit4] 2> 1606069 T4268 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: retry=false
: [junit4] 2> 1606070 T4268 oascc.SolrZkClient.makePath makePath: /overseer_elect
: [junit4] 2> 1606072 T4268 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
: [junit4] 2> 1606073 T4268 oasc.Overseer.close Overseer (id=null) closing
: [junit4] 2> 1606074 T4268 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:51454_solr
: [junit4] 2> 1606075 T4268 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
: [junit4] 2> 1606076 T4268 oasc.Overseer.start Overseer (id=92288351188877315-127.0.0.1:51454_solr-n_0000000000) starting
: [junit4] 2> 1606077 T4268 oascc.SolrZkClient.makePath makePath: /overseer/queue
: [junit4] 2> 1606079 T4268 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
: [junit4] 2> 1606080 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
: [junit4] 2> 1606082 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
: [junit4] 2> 1606084 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
: [junit4] 2> 1606086 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
: [junit4] 2> 1606089 T4268 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
: [junit4] 2> 1606089 T4305 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
: [junit4] 2> 1606090 T4268 oascc.SolrZkClient.makePath makePath: /clusterstate.json
: [junit4] 2> 1606090 T4304 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
: [junit4] 2> 1606092 T4268 oascc.SolrZkClient.makePath makePath: /aliases.json
: [junit4] 2> 1606093 T4268 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
: [junit4] 2> 1606094 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
: [junit4] 2> 1606096 T4308 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1e910c3 name:ZooKeeperConnection Watcher:127.0.0.1:51454/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 1606096 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
: [junit4] 2> 1606097 T4268 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
: [junit4] 2> 1606099 T4268 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1
: [junit4] 2> 1606104 T4311 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
: [junit4] 2> 1606106 T4304 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
: [junit4] 2> "operation":"state",
: [junit4] 2> "state":"active",
: [junit4] 2> "node_name":"127.0.0.1",
: [junit4] 2> "core":"core1",
: [junit4] 2> "core_node_name":"node1",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "numShards":"3",
: [junit4] 2> "base_url":"http://127.0.0.1/solr/"}
: [junit4] 2> 1606106 T4304 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2, shard3]
: [junit4] 2> 1606107 T4304 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
: [junit4] 2> 1606109 T4309 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 1606109 T4310 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 1606603 T4268 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
: [junit4] 2> 1606608 T4268 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
: [junit4] 2> 1606611 T4311 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
: [junit4] 2> 1606614 T4304 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
: [junit4] 2> "operation":"state",
: [junit4] 2> "state":"active",
: [junit4] 2> "node_name":"127.0.0.1",
: [junit4] 2> "core":"core2",
: [junit4] 2> "core_node_name":"node2",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "numShards":"3",
: [junit4] 2> "base_url":"http://127.0.0.1/solr/"}
: [junit4] 2> 1606614 T4304 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
: [junit4] 2> 1606614 T4304 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
: [junit4] 2> 1606615 T4311 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
: [junit4] 2> 1606616 T4309 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 1606616 T4310 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 1607112 T4268 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
: [junit4] 2> 1607117 T4268 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard3
: [junit4] 2> 1607121 T4311 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
: [junit4] 2> 1607124 T4304 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
: [junit4] 2> "operation":"state",
: [junit4] 2> "state":"active",
: [junit4] 2> "node_name":"127.0.0.1",
: [junit4] 2> "core":"core3",
: [junit4] 2> "core_node_name":"node3",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "numShards":"3",
: [junit4] 2> "base_url":"http://127.0.0.1/solr/"}
: [junit4] 2> 1607124 T4304 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
: [junit4] 2> 1607125 T4304 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
: [junit4] 2> 1607126 T4311 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
: [junit4] 2> 1607127 T4309 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 1607127 T4310 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 1607622 T4268 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
: [junit4] 2> 1607627 T4268 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
: [junit4] 2> 1607630 T4311 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
: [junit4] 2> 1607633 T4304 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
: [junit4] 2> "operation":"state",
: [junit4] 2> "state":"active",
: [junit4] 2> "node_name":"127.0.0.1",
: [junit4] 2> "core":"core4",
: [junit4] 2> "core_node_name":"node4",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "numShards":"3",
: [junit4] 2> "base_url":"http://127.0.0.1/solr/"}
: [junit4] 2> 1607633 T4304 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
: [junit4] 2> 1607633 T4304 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
: [junit4] 2> 1607635 T4311 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
: [junit4] 2> 1607636 T4310 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 1607636 T4309 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 1608131 T4268 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
: [junit4] 2> 1608137 T4311 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
: [junit4] 2> 1608138 T4304 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
: [junit4] 2> "operation":"state",
: [junit4] 2> "state":"active",
: [junit4] 2> "node_name":"127.0.0.1",
: [junit4] 2> "core":"core5",
: [junit4] 2> "core_node_name":"node5",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "numShards":"3",
: [junit4] 2> "base_url":"http://127.0.0.1/solr/"}
: [junit4] 2> 1608138 T4304 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
: [junit4] 2> 1608138 T4304 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
: [junit4] 2> 1608241 T4310 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 1608241 T4309 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 1608637 T4268 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
: [junit4] 2> 1608642 T4311 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
: [junit4] 2> 1608644 T4304 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
: [junit4] 2> "operation":"state",
: [junit4] 2> "state":"active",
: [junit4] 2> "node_name":"127.0.0.1",
: [junit4] 2> "core":"core6",
: [junit4] 2> "core_node_name":"node6",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "numShards":"3",
: [junit4] 2> "base_url":"http://127.0.0.1/solr/"}
: [junit4] 2> 1608644 T4304 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
: [junit4] 2> 1608644 T4304 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
: [junit4] 2> 1608746 T4310 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 1608746 T4309 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 1609142 T4268 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
: [junit4] 2> 1609148 T4292 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x147dfc435850004, likely client has closed socket
: [junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
: [junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
: [junit4] 2> at java.lang.Thread.run(Thread.java:745)
: [junit4] 2>
: [junit4] 2> 1609148 T4312 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard3/election
: [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
: [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
: [junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:290)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:287)
: [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:287)
: [junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
: [junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
: [junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:358)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:209)
: [junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
: [junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
: [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
: [junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
: [junit4] 2> at java.lang.Thread.run(Thread.java:745)
: [junit4] 2>
: [junit4] 2> 1609148 T4308 oaz.ClientCnxn$EventThread.processEvent ERROR Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@1a713ca rejected from java.util.concurrent.ThreadPoolExecutor@466e9[Shutting down, pool size = 2, active threads = 2, queued tasks = 0, completed tasks = 8]
: [junit4] 2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
: [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
: [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
: [junit4] 2> at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:110)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3.process(SolrZkClient.java:206)
: [junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
: [junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
: [junit4] 2>
: [junit4] 2> 1610648 T4310 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard2/election
: [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
: [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
: [junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:290)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:287)
: [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:287)
: [junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
: [junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
: [junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:358)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:209)
: [junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
: [junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
: [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
: [junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
: [junit4] 2> at java.lang.Thread.run(Thread.java:745)
: [junit4] 2>
: [junit4] 2> 1610650 T4268 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:51454 51454
: [junit4] 2> 1611085 T4291 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:51454 51454
: [junit4] 2> 1611086 T4268 oas.SolrTestCaseJ4.tearDown ###Ending testShardAssignment
: [junit4] 2> 1611087 T4268 oasc.Overseer.close Overseer (id=92288351188877315-127.0.0.1:51454_solr-n_0000000000) closing
: [junit4] 2> 1611087 T4304 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:51454_solr
: [junit4] 2> 1611088 T4313 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
: [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
: [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
: [junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
: [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
: [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:321)
: [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:89)
: [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:293)
: [junit4] 2>
: [junit4] 2> 1611093 T4268 oas.SolrTestCaseJ4.setUp ###Starting testReplay
: [junit4] 2> 1611093 T4268 oasc.ZkTestServer.run STARTING ZK TEST SERVER
: [junit4] 1> client port:0.0.0.0/0.0.0.0:0
: [junit4] 2> 1611094 T4314 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
: [junit4] 2> 1611194 T4268 oasc.ZkTestServer.run start zk server on port:50873
: [junit4] 2> 1611200 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
: [junit4] 2> 1611201 T4320 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f4feab name:ZooKeeperConnection Watcher:127.0.0.1:50873/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 1611202 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
: [junit4] 2> 1611206 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
: [junit4] 2> 1611207 T4322 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c8ac4b name:ZooKeeperConnection Watcher:127.0.0.1:50873 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 1611207 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
: [junit4] 2> 1611209 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
: [junit4] 2> 1611210 T4324 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@16d2b7d name:ZooKeeperConnection Watcher:127.0.0.1:50873 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 1611211 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
: [junit4] 2> 1611211 T4268 oascc.SolrZkClient.makePath makePath: /solr
: [junit4] 2> 1611213 T4268 oascc.SolrZkClient.makePath makePath: /live_nodes
: [junit4] 2> 1611215 T4268 oascc.SolrZkClient.makePath makePath: /clusterstate.json
: [junit4] 2> 1611216 T4268 oascc.SolrZkClient.makePath makePath: /aliases.json
: [junit4] 2> 1611218 T4268 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
: [junit4] 2> 1611220 T4268 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
: [junit4] 2> 1611223 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
: [junit4] 2> 1611223 T4326 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1ebe264 name:ZooKeeperConnection Watcher:127.0.0.1:50873/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 1611224 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
: [junit4] 2> 1611225 T4268 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: retry=false
: [junit4] 2> 1611225 T4268 oascc.SolrZkClient.makePath makePath: /overseer_elect
: [junit4] 2> 1611227 T4268 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
: [junit4] 2> 1611228 T4268 oasc.Overseer.close Overseer (id=null) closing
: [junit4] 2> 1611230 T4268 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:50873_solr
: [junit4] 2> 1611230 T4268 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
: [junit4] 2> 1611231 T4268 oasc.Overseer.start Overseer (id=92288351525666819-127.0.0.1:50873_solr-n_0000000000) starting
: [junit4] 2> 1611233 T4268 oascc.SolrZkClient.makePath makePath: /overseer/queue
: [junit4] 2> 1611236 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
: [junit4] 2> 1611238 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
: [junit4] 2> 1611240 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
: [junit4] 2> 1611243 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
: [junit4] 2> 1611247 T4268 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
: [junit4] 2> 1611248 T4328 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
: [junit4] 2> 1611252 T4327 oasc.Overseer$ClusterStateUpdater.run Replaying operations from work queue.
: [junit4] 2> 1611252 T4327 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
: [junit4] 2> "operation":"state",
: [junit4] 2> "base_url":"http://127.0.0.1/solr",
: [junit4] 2> "node_name":"node1",
: [junit4] 2> "shard":"s1",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "core":"core1",
: [junit4] 2> "roles":"",
: [junit4] 2> "state":"recovering"}
: [junit4] 2> 1611253 T4330 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
: [junit4] 2> 1611255 T4327 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
: [junit4] 2> "operation":"state",
: [junit4] 2> "base_url":"http://127.0.0.1/solr",
: [junit4] 2> "node_name":"node1",
: [junit4] 2> "shard":"s1",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "core":"core2",
: [junit4] 2> "roles":"",
: [junit4] 2> "state":"recovering"}
: [junit4] 2> 1611256 T4330 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
: [junit4] 2> 1611258 T4327 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
: [junit4] 2> 1611260 T4327 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
: [junit4] 2> "operation":"state",
: [junit4] 2> "base_url":"http://127.0.0.1/solr",
: [junit4] 2> "node_name":"node1",
: [junit4] 2> "shard":"s1",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "core":"core3",
: [junit4] 2> "roles":"",
: [junit4] 2> "state":"recovering"}
: [junit4] 2> 1611262 T4331 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
: [junit4] 2> 1611263 T4330 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
: [junit4] 2> 1611368 T4268 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:50873 50873
: [junit4] 2> 1611818 T4314 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:50873 50873
: [junit4] 2> 1611820 T4268 oas.SolrTestCaseJ4.tearDown ###Ending testReplay
: [junit4] 2> 1611820 T4268 oasc.Overseer.close Overseer (id=92288351525666819-127.0.0.1:50873_solr-n_0000000000) closing
: [junit4] 2> 1611821 T4327 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:50873_solr
: [junit4] 2> 1611823 T4332 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
: [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
: [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
: [junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
: [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
: [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:321)
: [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:89)
: [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:293)
: [junit4] 2>
: [junit4] 2> 1611829 T4268 oas.SolrTestCaseJ4.setUp ###Starting testOverseerFailure
: [junit4] 2> 1611829 T4268 oasc.ZkTestServer.run STARTING ZK TEST SERVER
: [junit4] 1> client port:0.0.0.0/0.0.0.0:0
: [junit4] 2> 1611830 T4333 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
: [junit4] 2> 1611930 T4268 oasc.ZkTestServer.run start zk server on port:40099
: [junit4] 2> 1611931 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
: [junit4] 2> 1611933 T4339 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ff693a name:ZooKeeperConnection Watcher:127.0.0.1:40099 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 1611933 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
: [junit4] 2> 1611935 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
: [junit4] 2> 1611936 T4341 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@12fac1c name:ZooKeeperConnection Watcher:127.0.0.1:40099 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 1611936 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
: [junit4] 2> 1611936 T4268 oascc.SolrZkClient.makePath makePath: /solr
: [junit4] 2> 1611939 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
: [junit4] 2> 1611940 T4343 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1df2969 name:ZooKeeperConnection Watcher:127.0.0.1:40099/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 1611940 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
: [junit4] 2> 1611941 T4268 oascc.SolrZkClient.makePath makePath: /live_nodes
: [junit4] 2> 1611942 T4268 oascc.SolrZkClient.makePath makePath: /clusterstate.json
: [junit4] 2> 1611943 T4268 oascc.SolrZkClient.makePath makePath: /aliases.json
: [junit4] 2> 1611944 T4268 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
: [junit4] 2> 1611946 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
: [junit4] 2> 1611947 T4345 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@12eded name:ZooKeeperConnection Watcher:127.0.0.1:40099/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 1611947 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
: [junit4] 2> 1611948 T4268 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
: [junit4] 2> 1611950 T4268 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
: [junit4] 2> 1611953 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
: [junit4] 2> 1611953 T4349 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@183bb88 name:ZooKeeperConnection Watcher:127.0.0.1:40099/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 1611954 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
: [junit4] 2> 1611954 T4268 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: retry=false
: [junit4] 2> 1611955 T4268 oascc.SolrZkClient.makePath makePath: /overseer_elect
: [junit4] 2> 1611956 T4268 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
: [junit4] 2> 1611957 T4268 oasc.Overseer.close Overseer (id=null) closing
: [junit4] 2> 1611959 T4268 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:40099_solr
: [junit4] 2> 1611959 T4268 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
: [junit4] 2> 1611960 T4268 oasc.Overseer.start Overseer (id=92288351573835780-127.0.0.1:40099_solr-n_0000000000) starting
: [junit4] 2> 1611961 T4268 oascc.SolrZkClient.makePath makePath: /overseer/queue
: [junit4] 2> 1611963 T4268 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
: [junit4] 2> 1611965 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
: [junit4] 2> 1611967 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
: [junit4] 2> 1611969 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
: [junit4] 2> 1611972 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
: [junit4] 2> 1611976 T4268 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
: [junit4] 2> 1611976 T4351 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
: [junit4] 2> 1611977 T4350 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
: [junit4] 2> 1612988 T4353 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
: [junit4] 2> 1612991 T4350 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
: [junit4] 2> "operation":"state",
: [junit4] 2> "state":"recovering",
: [junit4] 2> "node_name":"node1",
: [junit4] 2> "core":"core1",
: [junit4] 2> "core_node_name":"core_node1",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "numShards":"1",
: [junit4] 2> "base_url":"http://node1/solr/"}
: [junit4] 2> 1612991 T4350 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1]
: [junit4] 2> 1612991 T4350 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
: [junit4] 2> 1612993 T4347 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 1612993 T4346 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 1613488 T4268 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
: [junit4] 2> 1613496 T4268 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
: [junit4] 2> 1613501 T4353 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
: [junit4] 2> 1613504 T4268 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
: [junit4] 2> 1613505 T4350 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
: [junit4] 2> "operation":"state",
: [junit4] 2> "state":"active",
: [junit4] 2> "node_name":"node1",
: [junit4] 2> "core":"core1",
: [junit4] 2> "core_node_name":"core_node1",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "numShards":"1",
: [junit4] 2> "base_url":"http://node1/solr/"}
: [junit4] 2> 1613505 T4350 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
: [junit4] 2> 1613507 T4353 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
: [junit4] 2> 1613508 T4347 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 1613508 T4346 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 1613513 T4334 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x147dfc44c770004, likely client has closed socket
: [junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
: [junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
: [junit4] 2> at java.lang.Thread.run(Thread.java:745)
: [junit4] 2>
: [junit4] 2> 1613985 T4351 oasc.OverseerCollectionProcessor.amILeader org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
: [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
: [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
: [junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
: [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
: [junit4] 2> at org.apache.solr.cloud.OverseerCollectionProcessor.amILeader(OverseerCollectionProcessor.java:516)
: [junit4] 2> at org.apache.solr.cloud.OverseerCollectionProcessor.run(OverseerCollectionProcessor.java:271)
: [junit4] 2> at java.lang.Thread.run(Thread.java:745)
: [junit4] 2>
: [junit4] 2> 1613985 T4351 oasc.OverseerCollectionProcessor.amILeader According to ZK I (id=92288351573835780-127.0.0.1:40099_solr-n_0000000000) am no longer a leader.
: [junit4] 2> 1614515 T4268 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
: [junit4] 2> 1614519 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
: [junit4] 2> 1614520 T4355 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1dc1ae1 name:ZooKeeperConnection Watcher:127.0.0.1:40099/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 1614520 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
: [junit4] 2> 1614521 T4268 oasc.Overseer.close Overseer (id=92288351573835780-127.0.0.1:40099_solr-n_0000000000) closing
: [junit4] 2> 1614521 T4350 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:40099_solr
: [junit4] 2> 1614522 T4268 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: retry=false
: [junit4] 2> 1614523 T4356 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
: [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
: [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
: [junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
: [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
: [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:321)
: [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:89)
: [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:293)
: [junit4] 2>
: [junit4] 2> 1614524 T4268 oasc.Overseer.close Overseer (id=null) closing
: [junit4] 2> 1614525 T4268 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:40099_solr
: [junit4] 2> 1614525 T4268 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
: [junit4] 2> 1614527 T4268 oasc.Overseer.start Overseer (id=92288351573835781-127.0.0.1:40099_solr-n_0000000001) starting
: [junit4] 2> 1614532 T4268 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
: [junit4] 2> 1614532 T4358 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
: [junit4] 2> 1614534 T4357 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
: [junit4] 2> 1614535 T4357 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
: [junit4] 2> "operation":"state",
: [junit4] 2> "state":"recovering",
: [junit4] 2> "node_name":"node1",
: [junit4] 2> "core":"core1",
: [junit4] 2> "core_node_name":"core_node1",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "numShards":"1",
: [junit4] 2> "base_url":"http://node1/solr/"}
: [junit4] 2> 1614536 T4357 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
: [junit4] 2> 1614537 T4360 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
: [junit4] 2> 1614537 T4346 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 1614537 T4347 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
: [junit4] 2> 1614590 T4268 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
: [junit4] 2> 1614590 T4360 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
: [junit4] 2> 1614594 T4268 oasc.LeaderElector.checkIfIamLeader WARN Failed setting watch org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/collection1/leader_elect/shard1/election/92288351573835779-node1_core1-n_0000000002
: [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
: [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
: [junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
: [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
: [junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:148)
: [junit4] 2> at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:314)
: [junit4] 2> at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
: [junit4] 2> at org.apache.solr.cloud.OverseerTest$MockZKController.publishState(OverseerTest.java:157)
: [junit4] 2> at org.apache.solr.cloud.OverseerTest.testOverseerFailure(OverseerTest.java:662)
: [junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
: [junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
: [junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
: [junit4] 2> at java.lang.reflect.Method.invoke(Method.java:606)
: [junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
: [junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
: [junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
: [junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
: [junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
: [junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
: [junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
: [junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
: [junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
: [junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
: [junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
: [junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: [junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
: [junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
: [junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
: [junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
: [junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
: [junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
: [junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
: [junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: [junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
: [junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
: [junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
: [junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
: [junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
: [junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
: [junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: [junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: [junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: [junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
: [junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
: [junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
: [junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
: [junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: [junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
: [junit4] 2> at java.lang.Thread.run(Thread.java:745)
: [junit4] 2>
: [junit4] 2> 1614595 T4346 oasc.LeaderElector.checkIfIamLeader WARN Our node is no longer in line to be leader
: [junit4] 2> 1614596 T4361 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/collection1/leader_elect/shard1/election
: [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
: [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
: [junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:290)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:287)
: [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:287)
: [junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
: [junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
: [junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:358)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:209)
: [junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
: [junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
: [junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
: [junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
: [junit4] 2> at java.lang.Thread.run(Thread.java:745)
: [junit4] 2>
: [junit4] 2> 1614599 T4268 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:40099 40099
: [junit4] 2> 1615294 T4333 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:40099 40099
: [junit4] 2> 1615295 T4268 oas.SolrTestCaseJ4.tearDown ###Ending testOverseerFailure
: [junit4] 2> 1615295 T4268 oasc.Overseer.close Overseer (id=92288351573835781-127.0.0.1:40099_solr-n_0000000001) closing
: [junit4] 2> 1615297 T4357 oasc.SolrException.log ERROR Cleaning up collection in zk was interrupted:collection1:java.lang.InterruptedException: sleep interrupted
: [junit4] 2> at java.lang.Thread.sleep(Native Method)
: [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:119)
: [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:87)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:186)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.clean(SolrZkClient.java:670)
: [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.removeCore(Overseer.java:1098)
: [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.processMessage(Overseer.java:363)
: [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:246)
: [junit4] 2> at java.lang.Thread.run(Thread.java:745)
: [junit4] 2>
: [junit4] 2> 1615297 T4357 oasc.Overseer$ClusterStateUpdater.run WARN Solr cannot talk to ZK, exiting Overseer main queue loop org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer/queue-work/qn-
: [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
: [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
: [junit4] 2> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:273)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:270)
: [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:270)
: [junit4] 2> at org.apache.solr.cloud.DistributedQueue.createData(DistributedQueue.java:369)
: [junit4] 2> at org.apache.solr.cloud.DistributedQueue.offer(DistributedQueue.java:353)
: [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:259)
: [junit4] 2> at java.lang.Thread.run(Thread.java:745)
: [junit4] 2>
: [junit4] 2> 1615298 T4357 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:40099_solr
: [junit4] 2> 1615299 T4362 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
: [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
: [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
: [junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
: [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
: [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:321)
: [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:89)
: [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:293)
: [junit4] 2>
: [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=OverseerTest -Dtests.method=testOverseerFailure -Dtests.seed=5547E29057A024AB -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es_UY -Dtests.timezone=America/Santa_Isabel -Dtests.file.encoding=ISO-8859-1
: [junit4] ERROR 3.48s J0 | OverseerTest.testOverseerFailure <<<
: [junit4] > Throwable #1: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/collection1/leader_elect/shard1/election
: [junit4] > at __randomizedtesting.SeedInfo.seed([5547E29057A024AB:514F6D634505CB8A]:0)
: [junit4] > at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
: [junit4] > at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
: [junit4] > at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
: [junit4] > at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:290)
: [junit4] > at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:287)
: [junit4] > at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
: [junit4] > at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:287)
: [junit4] > at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
: [junit4] > at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:155)
: [junit4] > at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:314)
: [junit4] > at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
: [junit4] > at org.apache.solr.cloud.OverseerTest$MockZKController.publishState(OverseerTest.java:157)
: [junit4] > at org.apache.solr.cloud.OverseerTest.testOverseerFailure(OverseerTest.java:662)
: [junit4] > at java.lang.Thread.run(Thread.java:745)
: [junit4] 2> 1615312 T4268 oas.SolrTestCaseJ4.setUp ###Starting testStateChange
: [junit4] 2> 1615313 T4268 oasc.ZkTestServer.run STARTING ZK TEST SERVER
: [junit4] 1> client port:0.0.0.0/0.0.0.0:0
: [junit4] 2> 1615314 T4363 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
: [junit4] 2> 1615414 T4268 oasc.ZkTestServer.run start zk server on port:57017
: [junit4] 2> 1615415 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
: [junit4] 2> 1615417 T4369 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14ae008 name:ZooKeeperConnection Watcher:127.0.0.1:57017/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 1615417 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
: [junit4] 2> 1615420 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
: [junit4] 2> 1615421 T4371 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@172325d name:ZooKeeperConnection Watcher:127.0.0.1:57017 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 1615421 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
: [junit4] 2> 1615423 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
: [junit4] 2> 1615424 T4373 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1218652 name:ZooKeeperConnection Watcher:127.0.0.1:57017 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 1615424 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
: [junit4] 2> 1615425 T4268 oascc.SolrZkClient.makePath makePath: /solr
: [junit4] 2> 1615427 T4268 oascc.SolrZkClient.makePath makePath: /live_nodes
: [junit4] 2> 1615429 T4268 oascc.SolrZkClient.makePath makePath: /clusterstate.json
: [junit4] 2> 1615431 T4268 oascc.SolrZkClient.makePath makePath: /aliases.json
: [junit4] 2> 1615433 T4268 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
: [junit4] 2> 1615435 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
: [junit4] 2> 1615436 T4375 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@13f476c name:ZooKeeperConnection Watcher:127.0.0.1:57017/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 1615436 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
: [junit4] 2> 1615436 T4268 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: retry=false
: [junit4] 2> 1615437 T4268 oascc.SolrZkClient.makePath makePath: /overseer_elect
: [junit4] 2> 1615439 T4268 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
: [junit4] 2> 1615441 T4268 oasc.Overseer.close Overseer (id=null) closing
: [junit4] 2> 1615443 T4268 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:57017_solr
: [junit4] 2> 1615443 T4268 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
: [junit4] 2> 1615445 T4268 oasc.Overseer.start Overseer (id=92288351802163203-127.0.0.1:57017_solr-n_0000000000) starting
: [junit4] 2> 1615448 T4268 oascc.SolrZkClient.makePath makePath: /overseer/queue
: [junit4] 2> 1615451 T4268 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
: [junit4] 2> 1615452 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
: [junit4] 2> 1615454 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
: [junit4] 2> 1615456 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
: [junit4] 2> 1615458 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
: [junit4] 2> 1615460 T4268 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
: [junit4] 2> 1615461 T4377 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
: [junit4] 2> 1615462 T4376 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
: [junit4] 2> 1615464 T4376 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
: [junit4] 2> "operation":"state",
: [junit4] 2> "base_url":"http://127.0.0.1/solr",
: [junit4] 2> "node_name":"node1",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "core":"core1",
: [junit4] 2> "roles":"",
: [junit4] 2> "state":"recovering"}
: [junit4] 2> 1615464 T4376 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
: [junit4] 2> 1615466 T4379 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
: [junit4] 2> 1615466 T4380 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
: [junit4] 2> 1615515 T4379 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
: [junit4] 2> 1615516 T4376 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
: [junit4] 2> "operation":"state",
: [junit4] 2> "base_url":"http://127.0.0.1/solr",
: [junit4] 2> "node_name":"node1",
: [junit4] 2> "collection":"collection1",
: [junit4] 2> "core":"core1",
: [junit4] 2> "roles":"",
: [junit4] 2> "state":"active"}
: [junit4] 2> 1615517 T4376 oasc.Overseer$ClusterStateUpdater.updateState node=core_node1 is already registered
: [junit4] 2> 1615517 T4376 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
: [junit4] 2> 1615619 T4380 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
: [junit4] 2> 1615667 T4268 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:57017 57017
: [junit4] 2> 1616218 T4363 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:57017 57017
: [junit4] 2> 1616219 T4268 oas.SolrTestCaseJ4.tearDown ###Ending testStateChange
: [junit4] 2> 1616219 T4268 oasc.Overseer.close Overseer (id=92288351802163203-127.0.0.1:57017_solr-n_0000000000) closing
: [junit4] 2> 1616220 T4376 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:57017_solr
: [junit4] 2> 1616225 T4381 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
: [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
: [junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
: [junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
: [junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
: [junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
: [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:321)
: [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:89)
: [junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:293)
: [junit4] 2>
: [junit4] 2> 1616229 T4268 oas.SolrTestCaseJ4.setUp ###Starting testShardAssignmentBigger
: [junit4] 2> 1616230 T4268 oasc.ZkTestServer.run STARTING ZK TEST SERVER
: [junit4] 1> client port:0.0.0.0/0.0.0.0:0
: [junit4] 2> 1616231 T4382 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
: [junit4] 2> 1616331 T4268 oasc.ZkTestServer.run start zk server on port:56278
: [junit4] 2> 1616332 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
: [junit4] 2> 1616333 T4388 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3284a1 name:ZooKeeperConnection Watcher:127.0.0.1:56278 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 1616334 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
: [junit4] 2> 1616336 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
: [junit4] 2> 1616337 T4390 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1533f20 name:ZooKeeperConnection Watcher:127.0.0.1:56278 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 1616337 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
: [junit4] 2> 1616337 T4268 oascc.SolrZkClient.makePath makePath: /solr
: [junit4] 2> 1616340 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
: [junit4] 2> 1616341 T4392 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@68d232 name:ZooKeeperConnection Watcher:127.0.0.1:56278/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 1616341 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
: [junit4] 2> 1616341 T4268 oascc.SolrZkClient.makePath makePath: /live_nodes
: [junit4] 2> 1616343 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
: [junit4] 2> 1616344 T4394 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a1e22d name:ZooKeeperConnection Watcher:127.0.0.1:56278/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
: [junit4] 2> 1616344 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
: [junit4] 2> 1616344 T4268 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: retry=false
: [junit4] 2> 1616345 T4268 oascc.SolrZkClient.makePath makePath: /overseer_elect
: [junit4] 2> 1616346 T4268 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
: [junit4] 2> 1616348 T4268 oasc.Overseer.close Overseer (id=null) closing
: [junit4] 2> 1616349 T4268 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:56278_solr
: [junit4] 2> 1616349 T4268 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
: [junit4] 2> 1616350 T4268 oasc.Overseer.start Overseer (id=92288351862259715-127.0.0.1:56278_solr-n_0000000000) starting
: [junit4] 2> 1616351 T4268 oascc.SolrZkClient.makePath makePath: /overseer/queue
: [junit4] 2> 1616353 T4268 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
: [junit4] 2> 1616355 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
: [junit4] 2> 1616356 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-
:
: [...truncated too long message...]
:
: ,cumulative_errors=0,transaction_logs_total_size=17895,transaction_logs_total_number=2}
: [junit4] 2> 1876365 T4734 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
: [junit4] 2> 1876365 T4734 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
: [junit4] 2> 1876365 T4734 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
: [junit4] 2> 1876367 T4734 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
: [junit4] 2> 1876367 T4734 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
: [junit4] 2> 1876367 T4734 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.TestDistributedSearch-5547E29057A024AB-001/tempDir-001/shard0/data [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.TestDistributedSearch-5547E29057A024AB-001/tempDir-001/shard0/data;done=false>>]
: [junit4] 2> 1876368 T4734 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.TestDistributedSearch-5547E29057A024AB-001/tempDir-001/shard0/data
: [junit4] 2> 1876368 T4734 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.TestDistributedSearch-5547E29057A024AB-001/tempDir-001/shard0/data/index [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.TestDistributedSearch-5547E29057A024AB-001/tempDir-001/shard0/data/index;done=false>>]
: [junit4] 2> 1876368 T4734 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.TestDistributedSearch-5547E29057A024AB-001/tempDir-001/shard0/data/index
: [junit4] 2> 1876369 T4734 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
: [junit4] 2> 1881341 T4734 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
: [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestDistributedSearch -Dtests.method=testDistribSearch -Dtests.seed=5547E29057A024AB -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=de_CH -Dtests.timezone=Asia/Saigon -Dtests.file.encoding=ISO-8859-1
: [junit4] FAILURE 14.6s J1 | TestDistributedSearch.testDistribSearch <<<
: [junit4] > Throwable #1: junit.framework.AssertionFailedError: .facet_counts.facet_fields.a_n_tdt.1970-01-01T00:00:00Z!=2010-04-20T11:00:00Z (unordered or missing)
: [junit4] > at __randomizedtesting.SeedInfo.seed([5547E29057A024AB:D4A16C8820FF4497]:0)
: [junit4] > at junit.framework.Assert.fail(Assert.java:50)
: [junit4] > at org.apache.solr.BaseDistributedSearchTestCase.compareSolrResponses(BaseDistributedSearchTestCase.java:842)
: [junit4] > at org.apache.solr.BaseDistributedSearchTestCase.compareResponses(BaseDistributedSearchTestCase.java:861)
: [junit4] > at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:562)
: [junit4] > at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:542)
: [junit4] > at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:521)
: [junit4] > at org.apache.solr.TestDistributedSearch.doTest(TestDistributedSearch.java:180)
: [junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:875)
: [junit4] > at java.lang.Thread.run(Thread.java:745)
: [junit4] 2> 1881364 T4734 oas.SolrTestCaseJ4.deleteCore ###deleteCore
: [junit4] 2> NOTE: leaving temporary files on disk at: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.TestDistributedSearch-5547E29057A024AB-001
: [junit4] 2> NOTE: test params are: codec=Lucene410: {b_n_tdt=PostingsFormat(name=Direct), n_tl1=Pulsing41(freqCutoff=16 minBlockSize=9 maxBlockSize=66), a_i1=PostingsFormat(name=Direct), range_facet_l=PostingsFormat(name=Direct), multiDefault=Pulsing41(freqCutoff=16 minBlockSize=9 maxBlockSize=66), a_t=PostingsFormat(name=Direct), intDefault=PostingsFormat(name=Direct), other_tl1=PostingsFormat(name=Direct), n_tf1=PostingsFormat(name=Direct), id=PostingsFormat(name=Lucene41WithOrds), SubjectTerms_mfacet=PostingsFormat(name=Direct), n_tdt1=PostingsFormat(name=Direct), n_td1=Pulsing41(freqCutoff=16 minBlockSize=9 maxBlockSize=66), n_d1=PostingsFormat(name=Direct), lowerfilt=Pulsing41(freqCutoff=16 minBlockSize=9 maxBlockSize=66), n_ti1=PostingsFormat(name=Lucene41WithOrds), text=PostingsFormat(name=Memory doPackFST= false), timestamp=PostingsFormat(name=Direct), n_l1=PostingsFormat(name=Direct), foo_d=PostingsFormat(name=Lucene41WithOrds), a_n_tdt=PostingsFormat(name=Lucene41Wit
hOrds), lowerfilt1and2=Pulsing41(freqCutoff=16 minBlockSize=9 maxBlockSize=66), oddField_s=PostingsFormat(name=Memory doPackFST= false), n_dt1=Pulsing41(freqCutoff=16 minBlockSize=9 maxBlockSize=66), _version_=PostingsFormat(name=Direct), rnd_b=PostingsFormat(name=Lucene41WithOrds), foo_b=PostingsFormat(name=Memory doPackFST= false), foo_f=PostingsFormat(name=Memory doPackFST= false), n_f1=Pulsing41(freqCutoff=16 minBlockSize=9 maxBlockSize=66)}, docValues:{timestamp=DocValuesFormat(name=Memory)}, sim=DefaultSimilarity, locale=de_CH, timezone=Asia/Saigon
: [junit4] 2> NOTE: Linux 3.13.0-32-generic i386/Oracle Corporation 1.7.0_65 (32-bit)/cpus=8,threads=1,free=399639880,total=518979584
: [junit4] 2> NOTE: All tests run in this JVM: [SortByFunctionTest, BinaryUpdateRequestHandlerTest, CollectionsAPIDistributedZkTest, ConvertedLegacyTest, FieldAnalysisRequestHandlerTest, CursorPagingTest, OpenCloseCoreStressTest, HdfsBasicDistributedZkTest, OverseerRolesTest, LegacyHTMLStripCharFilterTest, AutoCommitTest, HdfsWriteToMultipleCollectionsTest, TestCollationField, TestFunctionQuery, TestBinaryField, TestSimpleQParserPlugin, UniqFieldsUpdateProcessorFactoryTest, DocValuesTest, PrimUtilsTest, URLClassifyProcessorTest, TestStressLucene, CoreAdminRequestStatusTest, DistributedFacetPivotLongTailTest, TestSweetSpotSimilarityFactory, HdfsCollectionsAPIDistributedZkTest, CurrencyFieldXmlFileTest, TestMissingGroups, TestFieldSortValues, OutputWriterTest, TestCloudInspectUtil, TestSchemaVersionResource, DebugComponentTest, RequestHandlersTest, TestGroupingSearch, TestCustomSort, ScriptEngineTest, TestDocBasedVersionConstraints, TestStandardQParsers, TermVectorComponentDistrib
utedTest, DateFieldTest, TestSearcherReuse, ChaosMonkeySafeLeaderTest, DirectSolrConnectionTest, CircularListTest, DirectUpdateHandlerOptimizeTest, TestLazyCores, TestAddFieldRealTimeGet, AnalysisErrorHandlingTest, BasicDistributedZk2Test, LeaderElectionIntegrationTest, TestImplicitCoreProperties, ExpressionTest, AddSchemaFieldsUpdateProcessorFactoryTest, TestReversedWildcardFilterFactory, TestCollationFieldDocValues, SolrInfoMBeanTest, DistributedQueryElevationComponentTest, TestNonNRTOpen, TestManagedResource, RAMDirectoryFactoryTest, TestSolrQueryParserDefaultOperatorResource, TestJoin, TestLeaderElectionZkExpiry, MigrateRouteKeyTest, TestClassNameShortening, CoreMergeIndexesAdminHandlerTest, SolrRequestParserTest, TestIndexingPerformance, DistributedDebugComponentTest, TestFiltering, UpdateRequestProcessorFactoryTest, CursorMarkTest, MultiTermTest, TestCharFilters, UUIDFieldTest, SolrIndexConfigTest, TestSolrQueryParserResource, DistribDocExpirationUpdateProcessorTest, TestInitQ
Parser, TestQueryTypes, TestIntervalFaceting, DistributedQueryComponentCustomSortTest, HardAutoCommitTest, SynonymTokenizerTest, SolrXmlInZkTest, SearchHandlerTest, TestSchemaSimilarityResource, SolrPluginUtilsTest, TestUpdate, TestRandomFaceting, TestExceedMaxTermLength, QueryParsingTest, TestSolrXml, TestZkChroot, TestHighlightDedupGrouping, TestSolrXmlPersistence, DistributedTermsComponentTest, SolrCoreTest, FileBasedSpellCheckerTest, AbstractAnalyticsStatsTest, AnalysisAfterCoreReloadTest, PolyFieldTest, TermVectorComponentTest, QueryElevationComponentTest, CacheHeaderTest, BasicDistributedZkTest, BasicZkTest, FullSolrCloudDistribCmdsTest, ZkSolrClientTest, TestRandomDVFaceting, ShardRoutingCustomTest, TestDistributedSearch]
: [junit4] Completed on J1 in 14.63s, 1 test, 1 failure <<< FAILURES!
:
: [...truncated 641 lines...]
: BUILD FAILED
: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:474: The following error occurred while executing this line:
: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:454: The following error occurred while executing this line:
: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:496: The following error occurred while executing this line:
: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1350: The following error occurred while executing this line:
: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:965: There were test failures: 430 suites, 1782 tests, 1 error, 1 failure, 40 ignored (19 assumptions)
:
: Total time: 93 minutes 17 seconds
: Build step 'Invoke Ant' marked build as failure
: [description-setter] Description set: Java: 32bit/jdk1.7.0_65 -server -XX:+UseConcMarkSweepGC
: Archiving artifacts
: Recording test results
: Email was triggered for: Failure - Any
: Sending email for trigger: Failure - Any
:
:
:
-Hoss
http://www.lucidworks.com/
[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.7.0_65) - Build #
11037 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/11037/
Java: 32bit/jdk1.7.0_65 -server -XX:+UseConcMarkSweepGC
2 tests failed.
FAILED: org.apache.solr.TestDistributedSearch.testDistribSearch
Error Message:
.facet_counts.facet_fields.a_n_tdt.1970-01-01T00:00:00Z!=2010-04-20T11:00:00Z (unordered or missing)
Stack Trace:
junit.framework.AssertionFailedError: .facet_counts.facet_fields.a_n_tdt.1970-01-01T00:00:00Z!=2010-04-20T11:00:00Z (unordered or missing)
at __randomizedtesting.SeedInfo.seed([5547E29057A024AB:D4A16C8820FF4497]:0)
at junit.framework.Assert.fail(Assert.java:50)
at org.apache.solr.BaseDistributedSearchTestCase.compareSolrResponses(BaseDistributedSearchTestCase.java:842)
at org.apache.solr.BaseDistributedSearchTestCase.compareResponses(BaseDistributedSearchTestCase.java:861)
at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:562)
at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:542)
at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:521)
at org.apache.solr.TestDistributedSearch.doTest(TestDistributedSearch.java:180)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:875)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
FAILED: org.apache.solr.cloud.OverseerTest.testOverseerFailure
Error Message:
KeeperErrorCode = NoNode for /collections/collection1/leader_elect/shard1/election
Stack Trace:
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/collection1/leader_elect/shard1/election
at __randomizedtesting.SeedInfo.seed([5547E29057A024AB:514F6D634505CB8A]:0)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:290)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:287)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:287)
at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:155)
at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:314)
at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
at org.apache.solr.cloud.OverseerTest$MockZKController.publishState(OverseerTest.java:157)
at org.apache.solr.cloud.OverseerTest.testOverseerFailure(OverseerTest.java:662)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 11366 lines...]
[junit4] Suite: org.apache.solr.cloud.OverseerTest
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./temp/solr.cloud.OverseerTest-5547E29057A024AB-001/init-core-data-001
[junit4] 2> 1604704 T4268 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (false)
[junit4] 2> 1604704 T4268 oas.SolrTestCaseJ4.initCore ####initCore
[junit4] 2> 1604704 T4268 oas.SolrTestCaseJ4.initCore ####initCore end
[junit4] 2> 1604707 T4268 oas.SolrTestCaseJ4.setUp ###Starting testPlaceholders
[junit4] 2> 1604708 T4268 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1604708 T4269 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 1604816 T4268 oasc.ZkTestServer.run start zk server on port:46180
[junit4] 2> 1604817 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1604820 T4275 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@cacff9 name:ZooKeeperConnection Watcher:127.0.0.1:46180/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1604820 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1604821 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1604822 T4277 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a5eef5 name:ZooKeeperConnection Watcher:127.0.0.1:46180 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1604822 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1604824 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1604825 T4279 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@9022ad name:ZooKeeperConnection Watcher:127.0.0.1:46180 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1604825 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1604825 T4268 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 1604828 T4268 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 1604830 T4268 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 1604831 T4268 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 1604832 T4268 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1604839 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1604840 T4281 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ada0d1 name:ZooKeeperConnection Watcher:127.0.0.1:46180/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1604840 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1604841 T4268 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1604842 T4268 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 1604851 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1604852 T4285 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e7e1e7 name:ZooKeeperConnection Watcher:127.0.0.1:46180/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1604852 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1604853 T4268 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: retry=false
[junit4] 2> 1604854 T4268 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 1604855 T4268 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 1604857 T4268 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1604858 T4268 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:46180_solr
[junit4] 2> 1604858 T4268 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 1604860 T4268 oasc.Overseer.start Overseer (id=92288351107088388-127.0.0.1:46180_solr-n_0000000000) starting
[junit4] 2> 1604861 T4268 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 1604862 T4268 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 1604864 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 1604866 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 1604868 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 1604870 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 1604873 T4268 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 1604873 T4287 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 1604874 T4286 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 1604876 T4286 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=12 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"12",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 1604876 T4286 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2, shard3, shard4, shard5, shard6, shard7, shard8, shard9, shard10, shard11, shard12]
[junit4] 2> 1604877 T4286 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 1604878 T4289 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1604879 T4282 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1604879 T4283 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1605376 T4268 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1605381 T4268 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1605385 T4289 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1605389 T4268 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:46180 46180
[junit4] 2> 1605947 T4269 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:46180 46180
[junit4] 2> 1605949 T4268 oas.SolrTestCaseJ4.tearDown ###Ending testPlaceholders
[junit4] 2> 1605949 T4268 oasc.Overseer.close Overseer (id=92288351107088388-127.0.0.1:46180_solr-n_0000000000) closing
[junit4] 2> 1605950 T4286 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:46180_solr
[junit4] 2> 1605951 T4290 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:321)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:89)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:293)
[junit4] 2>
[junit4] 2> 1605955 T4268 oas.SolrTestCaseJ4.setUp ###Starting testShardAssignment
[junit4] 2> 1605955 T4268 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1605956 T4291 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 1606056 T4268 oasc.ZkTestServer.run start zk server on port:51454
[junit4] 2> 1606057 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1606059 T4297 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5c931c name:ZooKeeperConnection Watcher:127.0.0.1:51454 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1606059 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1606061 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1606062 T4299 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@cca746 name:ZooKeeperConnection Watcher:127.0.0.1:51454 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1606062 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1606063 T4268 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 1606065 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1606066 T4301 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@315e42 name:ZooKeeperConnection Watcher:127.0.0.1:51454/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1606066 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1606067 T4268 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 1606068 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1606069 T4303 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@775b26 name:ZooKeeperConnection Watcher:127.0.0.1:51454/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1606069 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1606069 T4268 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: retry=false
[junit4] 2> 1606070 T4268 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 1606072 T4268 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 1606073 T4268 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1606074 T4268 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:51454_solr
[junit4] 2> 1606075 T4268 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 1606076 T4268 oasc.Overseer.start Overseer (id=92288351188877315-127.0.0.1:51454_solr-n_0000000000) starting
[junit4] 2> 1606077 T4268 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 1606079 T4268 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 1606080 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 1606082 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 1606084 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 1606086 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 1606089 T4268 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 1606089 T4305 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 1606090 T4268 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 1606090 T4304 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 1606092 T4268 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 1606093 T4268 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1606094 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1606096 T4308 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1e910c3 name:ZooKeeperConnection Watcher:127.0.0.1:51454/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1606096 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1606097 T4268 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1606099 T4268 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1
[junit4] 2> 1606104 T4311 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1606106 T4304 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 1606106 T4304 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2, shard3]
[junit4] 2> 1606107 T4304 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 1606109 T4309 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1606109 T4310 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1606603 T4268 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1606608 T4268 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1606611 T4311 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1606614 T4304 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core2",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 1606614 T4304 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 1606614 T4304 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
[junit4] 2> 1606615 T4311 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1606616 T4309 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1606616 T4310 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1607112 T4268 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
[junit4] 2> 1607117 T4268 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard3
[junit4] 2> 1607121 T4311 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1607124 T4304 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core3",
[junit4] 2> "core_node_name":"node3",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 1607124 T4304 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 1607125 T4304 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 1607126 T4311 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1607127 T4309 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1607127 T4310 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1607622 T4268 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 1607627 T4268 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 1607630 T4311 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1607633 T4304 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core4",
[junit4] 2> "core_node_name":"node4",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 1607633 T4304 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 1607633 T4304 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 1607635 T4311 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1607636 T4310 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1607636 T4309 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1608131 T4268 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1608137 T4311 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1608138 T4304 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core5",
[junit4] 2> "core_node_name":"node5",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 1608138 T4304 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 1608138 T4304 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
[junit4] 2> 1608241 T4310 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1608241 T4309 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1608637 T4268 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
[junit4] 2> 1608642 T4311 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1608644 T4304 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core6",
[junit4] 2> "core_node_name":"node6",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 1608644 T4304 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 1608644 T4304 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 1608746 T4310 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1608746 T4309 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1609142 T4268 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 1609148 T4292 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x147dfc435850004, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 1609148 T4312 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard3/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:290)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:287)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:287)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:358)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:209)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 1609148 T4308 oaz.ClientCnxn$EventThread.processEvent ERROR Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@1a713ca rejected from java.util.concurrent.ThreadPoolExecutor@466e9[Shutting down, pool size = 2, active threads = 2, queued tasks = 0, completed tasks = 8]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
[junit4] 2> at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:110)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3.process(SolrZkClient.java:206)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] 2> 1610648 T4310 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard2/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:290)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:287)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:287)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:358)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:209)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 1610650 T4268 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:51454 51454
[junit4] 2> 1611085 T4291 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:51454 51454
[junit4] 2> 1611086 T4268 oas.SolrTestCaseJ4.tearDown ###Ending testShardAssignment
[junit4] 2> 1611087 T4268 oasc.Overseer.close Overseer (id=92288351188877315-127.0.0.1:51454_solr-n_0000000000) closing
[junit4] 2> 1611087 T4304 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:51454_solr
[junit4] 2> 1611088 T4313 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:321)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:89)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:293)
[junit4] 2>
[junit4] 2> 1611093 T4268 oas.SolrTestCaseJ4.setUp ###Starting testReplay
[junit4] 2> 1611093 T4268 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1611094 T4314 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 1611194 T4268 oasc.ZkTestServer.run start zk server on port:50873
[junit4] 2> 1611200 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1611201 T4320 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f4feab name:ZooKeeperConnection Watcher:127.0.0.1:50873/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1611202 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1611206 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1611207 T4322 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c8ac4b name:ZooKeeperConnection Watcher:127.0.0.1:50873 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1611207 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1611209 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1611210 T4324 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@16d2b7d name:ZooKeeperConnection Watcher:127.0.0.1:50873 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1611211 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1611211 T4268 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 1611213 T4268 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 1611215 T4268 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 1611216 T4268 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 1611218 T4268 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1611220 T4268 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 1611223 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1611223 T4326 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1ebe264 name:ZooKeeperConnection Watcher:127.0.0.1:50873/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1611224 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1611225 T4268 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: retry=false
[junit4] 2> 1611225 T4268 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 1611227 T4268 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 1611228 T4268 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1611230 T4268 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:50873_solr
[junit4] 2> 1611230 T4268 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 1611231 T4268 oasc.Overseer.start Overseer (id=92288351525666819-127.0.0.1:50873_solr-n_0000000000) starting
[junit4] 2> 1611233 T4268 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 1611236 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 1611238 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 1611240 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 1611243 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 1611247 T4268 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 1611248 T4328 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 1611252 T4327 oasc.Overseer$ClusterStateUpdater.run Replaying operations from work queue.
[junit4] 2> 1611252 T4327 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "shard":"s1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core1",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"}
[junit4] 2> 1611253 T4330 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 1611255 T4327 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "shard":"s1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core2",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"}
[junit4] 2> 1611256 T4330 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 1611258 T4327 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 1611260 T4327 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "shard":"s1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core3",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"}
[junit4] 2> 1611262 T4331 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1611263 T4330 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 1611368 T4268 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:50873 50873
[junit4] 2> 1611818 T4314 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:50873 50873
[junit4] 2> 1611820 T4268 oas.SolrTestCaseJ4.tearDown ###Ending testReplay
[junit4] 2> 1611820 T4268 oasc.Overseer.close Overseer (id=92288351525666819-127.0.0.1:50873_solr-n_0000000000) closing
[junit4] 2> 1611821 T4327 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:50873_solr
[junit4] 2> 1611823 T4332 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:321)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:89)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:293)
[junit4] 2>
[junit4] 2> 1611829 T4268 oas.SolrTestCaseJ4.setUp ###Starting testOverseerFailure
[junit4] 2> 1611829 T4268 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1611830 T4333 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 1611930 T4268 oasc.ZkTestServer.run start zk server on port:40099
[junit4] 2> 1611931 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1611933 T4339 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ff693a name:ZooKeeperConnection Watcher:127.0.0.1:40099 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1611933 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1611935 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1611936 T4341 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@12fac1c name:ZooKeeperConnection Watcher:127.0.0.1:40099 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1611936 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1611936 T4268 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 1611939 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1611940 T4343 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1df2969 name:ZooKeeperConnection Watcher:127.0.0.1:40099/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1611940 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1611941 T4268 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 1611942 T4268 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 1611943 T4268 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 1611944 T4268 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1611946 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1611947 T4345 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@12eded name:ZooKeeperConnection Watcher:127.0.0.1:40099/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1611947 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1611948 T4268 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1611950 T4268 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 1611953 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1611953 T4349 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@183bb88 name:ZooKeeperConnection Watcher:127.0.0.1:40099/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1611954 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1611954 T4268 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: retry=false
[junit4] 2> 1611955 T4268 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 1611956 T4268 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 1611957 T4268 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1611959 T4268 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:40099_solr
[junit4] 2> 1611959 T4268 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 1611960 T4268 oasc.Overseer.start Overseer (id=92288351573835780-127.0.0.1:40099_solr-n_0000000000) starting
[junit4] 2> 1611961 T4268 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 1611963 T4268 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 1611965 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 1611967 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 1611969 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 1611972 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 1611976 T4268 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 1611976 T4351 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 1611977 T4350 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 1612988 T4353 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1612991 T4350 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 1612991 T4350 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1]
[junit4] 2> 1612991 T4350 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 1612993 T4347 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1612993 T4346 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1613488 T4268 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1613496 T4268 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1613501 T4353 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1613504 T4268 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1613505 T4350 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 1613505 T4350 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 1613507 T4353 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1613508 T4347 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1613508 T4346 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1613513 T4334 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x147dfc44c770004, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 1613985 T4351 oasc.OverseerCollectionProcessor.amILeader org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.cloud.OverseerCollectionProcessor.amILeader(OverseerCollectionProcessor.java:516)
[junit4] 2> at org.apache.solr.cloud.OverseerCollectionProcessor.run(OverseerCollectionProcessor.java:271)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 1613985 T4351 oasc.OverseerCollectionProcessor.amILeader According to ZK I (id=92288351573835780-127.0.0.1:40099_solr-n_0000000000) am no longer a leader.
[junit4] 2> 1614515 T4268 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1614519 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1614520 T4355 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1dc1ae1 name:ZooKeeperConnection Watcher:127.0.0.1:40099/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1614520 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1614521 T4268 oasc.Overseer.close Overseer (id=92288351573835780-127.0.0.1:40099_solr-n_0000000000) closing
[junit4] 2> 1614521 T4350 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:40099_solr
[junit4] 2> 1614522 T4268 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: retry=false
[junit4] 2> 1614523 T4356 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:321)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:89)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:293)
[junit4] 2>
[junit4] 2> 1614524 T4268 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1614525 T4268 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:40099_solr
[junit4] 2> 1614525 T4268 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 1614527 T4268 oasc.Overseer.start Overseer (id=92288351573835781-127.0.0.1:40099_solr-n_0000000001) starting
[junit4] 2> 1614532 T4268 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 1614532 T4358 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 1614534 T4357 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 1614535 T4357 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"core_node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"1",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 1614536 T4357 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 1614537 T4360 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1614537 T4346 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1614537 T4347 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1614590 T4268 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1614590 T4360 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1614594 T4268 oasc.LeaderElector.checkIfIamLeader WARN Failed setting watch org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/collection1/leader_elect/shard1/election/92288351573835779-node1_core1-n_0000000002
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:148)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:314)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
[junit4] 2> at org.apache.solr.cloud.OverseerTest$MockZKController.publishState(OverseerTest.java:157)
[junit4] 2> at org.apache.solr.cloud.OverseerTest.testOverseerFailure(OverseerTest.java:662)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] 2> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[junit4] 2> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] 2> at java.lang.reflect.Method.invoke(Method.java:606)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4] 2> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4] 2> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
[junit4] 2> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4] 2> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4] 2> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4] 2> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
[junit4] 2> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 1614595 T4346 oasc.LeaderElector.checkIfIamLeader WARN Our node is no longer in line to be leader
[junit4] 2> 1614596 T4361 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/collection1/leader_elect/shard1/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:290)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:287)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:287)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:358)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:209)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 1614599 T4268 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:40099 40099
[junit4] 2> 1615294 T4333 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:40099 40099
[junit4] 2> 1615295 T4268 oas.SolrTestCaseJ4.tearDown ###Ending testOverseerFailure
[junit4] 2> 1615295 T4268 oasc.Overseer.close Overseer (id=92288351573835781-127.0.0.1:40099_solr-n_0000000001) closing
[junit4] 2> 1615297 T4357 oasc.SolrException.log ERROR Cleaning up collection in zk was interrupted:collection1:java.lang.InterruptedException: sleep interrupted
[junit4] 2> at java.lang.Thread.sleep(Native Method)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:119)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:87)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:186)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.clean(SolrZkClient.java:670)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.removeCore(Overseer.java:1098)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.processMessage(Overseer.java:363)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:246)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 1615297 T4357 oasc.Overseer$ClusterStateUpdater.run WARN Solr cannot talk to ZK, exiting Overseer main queue loop org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer/queue-work/qn-
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:273)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:270)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:270)
[junit4] 2> at org.apache.solr.cloud.DistributedQueue.createData(DistributedQueue.java:369)
[junit4] 2> at org.apache.solr.cloud.DistributedQueue.offer(DistributedQueue.java:353)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:259)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 1615298 T4357 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:40099_solr
[junit4] 2> 1615299 T4362 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:321)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:89)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:293)
[junit4] 2>
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=OverseerTest -Dtests.method=testOverseerFailure -Dtests.seed=5547E29057A024AB -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es_UY -Dtests.timezone=America/Santa_Isabel -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 3.48s J0 | OverseerTest.testOverseerFailure <<<
[junit4] > Throwable #1: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/collection1/leader_elect/shard1/election
[junit4] > at __randomizedtesting.SeedInfo.seed([5547E29057A024AB:514F6D634505CB8A]:0)
[junit4] > at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
[junit4] > at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] > at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:290)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:287)
[junit4] > at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:287)
[junit4] > at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
[junit4] > at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:155)
[junit4] > at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:314)
[junit4] > at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
[junit4] > at org.apache.solr.cloud.OverseerTest$MockZKController.publishState(OverseerTest.java:157)
[junit4] > at org.apache.solr.cloud.OverseerTest.testOverseerFailure(OverseerTest.java:662)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 1615312 T4268 oas.SolrTestCaseJ4.setUp ###Starting testStateChange
[junit4] 2> 1615313 T4268 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1615314 T4363 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 1615414 T4268 oasc.ZkTestServer.run start zk server on port:57017
[junit4] 2> 1615415 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1615417 T4369 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14ae008 name:ZooKeeperConnection Watcher:127.0.0.1:57017/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1615417 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1615420 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1615421 T4371 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@172325d name:ZooKeeperConnection Watcher:127.0.0.1:57017 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1615421 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1615423 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1615424 T4373 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1218652 name:ZooKeeperConnection Watcher:127.0.0.1:57017 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1615424 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1615425 T4268 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 1615427 T4268 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 1615429 T4268 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 1615431 T4268 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 1615433 T4268 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1615435 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1615436 T4375 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@13f476c name:ZooKeeperConnection Watcher:127.0.0.1:57017/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1615436 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1615436 T4268 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: retry=false
[junit4] 2> 1615437 T4268 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 1615439 T4268 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 1615441 T4268 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1615443 T4268 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:57017_solr
[junit4] 2> 1615443 T4268 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 1615445 T4268 oasc.Overseer.start Overseer (id=92288351802163203-127.0.0.1:57017_solr-n_0000000000) starting
[junit4] 2> 1615448 T4268 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 1615451 T4268 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 1615452 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 1615454 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 1615456 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 1615458 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 1615460 T4268 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 1615461 T4377 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 1615462 T4376 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 1615464 T4376 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core1",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"}
[junit4] 2> 1615464 T4376 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 1615466 T4379 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1615466 T4380 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 1615515 T4379 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1615516 T4376 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core1",
[junit4] 2> "roles":"",
[junit4] 2> "state":"active"}
[junit4] 2> 1615517 T4376 oasc.Overseer$ClusterStateUpdater.updateState node=core_node1 is already registered
[junit4] 2> 1615517 T4376 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 1615619 T4380 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 1615667 T4268 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:57017 57017
[junit4] 2> 1616218 T4363 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:57017 57017
[junit4] 2> 1616219 T4268 oas.SolrTestCaseJ4.tearDown ###Ending testStateChange
[junit4] 2> 1616219 T4268 oasc.Overseer.close Overseer (id=92288351802163203-127.0.0.1:57017_solr-n_0000000000) closing
[junit4] 2> 1616220 T4376 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:57017_solr
[junit4] 2> 1616225 T4381 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:321)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:89)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:293)
[junit4] 2>
[junit4] 2> 1616229 T4268 oas.SolrTestCaseJ4.setUp ###Starting testShardAssignmentBigger
[junit4] 2> 1616230 T4268 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1616231 T4382 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 1616331 T4268 oasc.ZkTestServer.run start zk server on port:56278
[junit4] 2> 1616332 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1616333 T4388 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3284a1 name:ZooKeeperConnection Watcher:127.0.0.1:56278 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1616334 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1616336 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1616337 T4390 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1533f20 name:ZooKeeperConnection Watcher:127.0.0.1:56278 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1616337 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1616337 T4268 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 1616340 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1616341 T4392 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@68d232 name:ZooKeeperConnection Watcher:127.0.0.1:56278/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1616341 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1616341 T4268 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 1616343 T4268 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1616344 T4394 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a1e22d name:ZooKeeperConnection Watcher:127.0.0.1:56278/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1616344 T4268 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1616344 T4268 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: retry=false
[junit4] 2> 1616345 T4268 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 1616346 T4268 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 1616348 T4268 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1616349 T4268 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:56278_solr
[junit4] 2> 1616349 T4268 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 1616350 T4268 oasc.Overseer.start Overseer (id=92288351862259715-127.0.0.1:56278_solr-n_0000000000) starting
[junit4] 2> 1616351 T4268 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 1616353 T4268 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 1616355 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 1616356 T4268 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-
[...truncated too long message...]
,cumulative_errors=0,transaction_logs_total_size=17895,transaction_logs_total_number=2}
[junit4] 2> 1876365 T4734 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4] 2> 1876365 T4734 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4] 2> 1876365 T4734 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4] 2> 1876367 T4734 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4] 2> 1876367 T4734 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4] 2> 1876367 T4734 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.TestDistributedSearch-5547E29057A024AB-001/tempDir-001/shard0/data [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.TestDistributedSearch-5547E29057A024AB-001/tempDir-001/shard0/data;done=false>>]
[junit4] 2> 1876368 T4734 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.TestDistributedSearch-5547E29057A024AB-001/tempDir-001/shard0/data
[junit4] 2> 1876368 T4734 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.TestDistributedSearch-5547E29057A024AB-001/tempDir-001/shard0/data/index [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.TestDistributedSearch-5547E29057A024AB-001/tempDir-001/shard0/data/index;done=false>>]
[junit4] 2> 1876368 T4734 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.TestDistributedSearch-5547E29057A024AB-001/tempDir-001/shard0/data/index
[junit4] 2> 1876369 T4734 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
[junit4] 2> 1881341 T4734 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestDistributedSearch -Dtests.method=testDistribSearch -Dtests.seed=5547E29057A024AB -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=de_CH -Dtests.timezone=Asia/Saigon -Dtests.file.encoding=ISO-8859-1
[junit4] FAILURE 14.6s J1 | TestDistributedSearch.testDistribSearch <<<
[junit4] > Throwable #1: junit.framework.AssertionFailedError: .facet_counts.facet_fields.a_n_tdt.1970-01-01T00:00:00Z!=2010-04-20T11:00:00Z (unordered or missing)
[junit4] > at __randomizedtesting.SeedInfo.seed([5547E29057A024AB:D4A16C8820FF4497]:0)
[junit4] > at junit.framework.Assert.fail(Assert.java:50)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase.compareSolrResponses(BaseDistributedSearchTestCase.java:842)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase.compareResponses(BaseDistributedSearchTestCase.java:861)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:562)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:542)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase.query(BaseDistributedSearchTestCase.java:521)
[junit4] > at org.apache.solr.TestDistributedSearch.doTest(TestDistributedSearch.java:180)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:875)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> 1881364 T4734 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.TestDistributedSearch-5547E29057A024AB-001
[junit4] 2> NOTE: test params are: codec=Lucene410: {b_n_tdt=PostingsFormat(name=Direct), n_tl1=Pulsing41(freqCutoff=16 minBlockSize=9 maxBlockSize=66), a_i1=PostingsFormat(name=Direct), range_facet_l=PostingsFormat(name=Direct), multiDefault=Pulsing41(freqCutoff=16 minBlockSize=9 maxBlockSize=66), a_t=PostingsFormat(name=Direct), intDefault=PostingsFormat(name=Direct), other_tl1=PostingsFormat(name=Direct), n_tf1=PostingsFormat(name=Direct), id=PostingsFormat(name=Lucene41WithOrds), SubjectTerms_mfacet=PostingsFormat(name=Direct), n_tdt1=PostingsFormat(name=Direct), n_td1=Pulsing41(freqCutoff=16 minBlockSize=9 maxBlockSize=66), n_d1=PostingsFormat(name=Direct), lowerfilt=Pulsing41(freqCutoff=16 minBlockSize=9 maxBlockSize=66), n_ti1=PostingsFormat(name=Lucene41WithOrds), text=PostingsFormat(name=Memory doPackFST= false), timestamp=PostingsFormat(name=Direct), n_l1=PostingsFormat(name=Direct), foo_d=PostingsFormat(name=Lucene41WithOrds), a_n_tdt=PostingsFormat(name=Lucene41WithOrds), lowerfilt1and2=Pulsing41(freqCutoff=16 minBlockSize=9 maxBlockSize=66), oddField_s=PostingsFormat(name=Memory doPackFST= false), n_dt1=Pulsing41(freqCutoff=16 minBlockSize=9 maxBlockSize=66), _version_=PostingsFormat(name=Direct), rnd_b=PostingsFormat(name=Lucene41WithOrds), foo_b=PostingsFormat(name=Memory doPackFST= false), foo_f=PostingsFormat(name=Memory doPackFST= false), n_f1=Pulsing41(freqCutoff=16 minBlockSize=9 maxBlockSize=66)}, docValues:{timestamp=DocValuesFormat(name=Memory)}, sim=DefaultSimilarity, locale=de_CH, timezone=Asia/Saigon
[junit4] 2> NOTE: Linux 3.13.0-32-generic i386/Oracle Corporation 1.7.0_65 (32-bit)/cpus=8,threads=1,free=399639880,total=518979584
[junit4] 2> NOTE: All tests run in this JVM: [SortByFunctionTest, BinaryUpdateRequestHandlerTest, CollectionsAPIDistributedZkTest, ConvertedLegacyTest, FieldAnalysisRequestHandlerTest, CursorPagingTest, OpenCloseCoreStressTest, HdfsBasicDistributedZkTest, OverseerRolesTest, LegacyHTMLStripCharFilterTest, AutoCommitTest, HdfsWriteToMultipleCollectionsTest, TestCollationField, TestFunctionQuery, TestBinaryField, TestSimpleQParserPlugin, UniqFieldsUpdateProcessorFactoryTest, DocValuesTest, PrimUtilsTest, URLClassifyProcessorTest, TestStressLucene, CoreAdminRequestStatusTest, DistributedFacetPivotLongTailTest, TestSweetSpotSimilarityFactory, HdfsCollectionsAPIDistributedZkTest, CurrencyFieldXmlFileTest, TestMissingGroups, TestFieldSortValues, OutputWriterTest, TestCloudInspectUtil, TestSchemaVersionResource, DebugComponentTest, RequestHandlersTest, TestGroupingSearch, TestCustomSort, ScriptEngineTest, TestDocBasedVersionConstraints, TestStandardQParsers, TermVectorComponentDistributedTest, DateFieldTest, TestSearcherReuse, ChaosMonkeySafeLeaderTest, DirectSolrConnectionTest, CircularListTest, DirectUpdateHandlerOptimizeTest, TestLazyCores, TestAddFieldRealTimeGet, AnalysisErrorHandlingTest, BasicDistributedZk2Test, LeaderElectionIntegrationTest, TestImplicitCoreProperties, ExpressionTest, AddSchemaFieldsUpdateProcessorFactoryTest, TestReversedWildcardFilterFactory, TestCollationFieldDocValues, SolrInfoMBeanTest, DistributedQueryElevationComponentTest, TestNonNRTOpen, TestManagedResource, RAMDirectoryFactoryTest, TestSolrQueryParserDefaultOperatorResource, TestJoin, TestLeaderElectionZkExpiry, MigrateRouteKeyTest, TestClassNameShortening, CoreMergeIndexesAdminHandlerTest, SolrRequestParserTest, TestIndexingPerformance, DistributedDebugComponentTest, TestFiltering, UpdateRequestProcessorFactoryTest, CursorMarkTest, MultiTermTest, TestCharFilters, UUIDFieldTest, SolrIndexConfigTest, TestSolrQueryParserResource, DistribDocExpirationUpdateProcessorTest, TestInitQParser, TestQueryTypes, TestIntervalFaceting, DistributedQueryComponentCustomSortTest, HardAutoCommitTest, SynonymTokenizerTest, SolrXmlInZkTest, SearchHandlerTest, TestSchemaSimilarityResource, SolrPluginUtilsTest, TestUpdate, TestRandomFaceting, TestExceedMaxTermLength, QueryParsingTest, TestSolrXml, TestZkChroot, TestHighlightDedupGrouping, TestSolrXmlPersistence, DistributedTermsComponentTest, SolrCoreTest, FileBasedSpellCheckerTest, AbstractAnalyticsStatsTest, AnalysisAfterCoreReloadTest, PolyFieldTest, TermVectorComponentTest, QueryElevationComponentTest, CacheHeaderTest, BasicDistributedZkTest, BasicZkTest, FullSolrCloudDistribCmdsTest, ZkSolrClientTest, TestRandomDVFaceting, ShardRoutingCustomTest, TestDistributedSearch]
[junit4] Completed on J1 in 14.63s, 1 test, 1 failure <<< FAILURES!
[...truncated 641 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:474: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:454: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:496: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1350: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:965: There were test failures: 430 suites, 1782 tests, 1 error, 1 failure, 40 ignored (19 assumptions)
Total time: 93 minutes 17 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 32bit/jdk1.7.0_65 -server -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Re: [JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.8.0_11) - Build #
11036 - Still Failing!
Posted by "david.w.smiley@gmail.com" <da...@gmail.com>.
I’ll look into it.
~ David Smiley
Freelance Apache Lucene/Solr Search Consultant/Developer
http://www.linkedin.com/in/davidwsmiley
On Sat, Aug 16, 2014 at 10:16 AM, Policeman Jenkins Server <
jenkins@thetaphi.de> wrote:
> Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/11036/
> Java: 64bit/jdk1.8.0_11 -XX:+UseCompressedOops -XX:+UseG1GC
>
> 1 tests failed.
> FAILED: org.apache.lucene.spatial.bbox.TestBBoxStrategy.testOperations
> {#19 seed=[631220E490F0DB53:EFFB6AF935D602D2]}
>
> Error Message:
> [BBoxWithin] Shouldn't match
> I#0:Rect(minX=180.0,maxX=180.0,minY=30.0,maxY=40.0)
> Q:Rect(minX=-180.0,maxX=-180.0,minY=-70.0,maxY=50.0)
>
> Stack Trace:
> java.lang.AssertionError: [BBoxWithin] Shouldn't match
> I#0:Rect(minX=180.0,maxX=180.0,minY=30.0,maxY=40.0)
> Q:Rect(minX=-180.0,maxX=-180.0,minY=-70.0,maxY=50.0)
> at
> __randomizedtesting.SeedInfo.seed([631220E490F0DB53:EFFB6AF935D602D2]:0)
> at org.junit.Assert.fail(Assert.java:93)
> at
> org.apache.lucene.spatial.prefix.RandomSpatialOpStrategyTestCase.fail(RandomSpatialOpStrategyTestCase.java:126)
> at
> org.apache.lucene.spatial.prefix.RandomSpatialOpStrategyTestCase.testOperation(RandomSpatialOpStrategyTestCase.java:115)
> at
> org.apache.lucene.spatial.prefix.RandomSpatialOpStrategyTestCase.testOperationRandomShapes(RandomSpatialOpStrategyTestCase.java:62)
> at
> org.apache.lucene.spatial.bbox.TestBBoxStrategy.testOperations(TestBBoxStrategy.java:109)
> at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:483)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
> at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
> at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
> at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
> at java.lang.Thread.run(Thread.java:745)
>
>
>
>
> Build Log:
> [...truncated 9686 lines...]
> [junit4] Suite: org.apache.lucene.spatial.bbox.TestBBoxStrategy
> [junit4] 2> أغس 16, 2014 9:16:10 ص
> org.apache.lucene.spatial.StrategyTestCase executeQueries
> [junit4] 2> INFO: testing queried for strategy BBoxStrategy
> field:bbox ctx=SpatialContext.GEO
> [junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestBBoxStrategy -Dtests.method=testOperations
> -Dtests.seed=631220E490F0DB53 -Dtests.multiplier=3 -Dtests.slow=true
> -Dtests.locale=ar_DZ -Dtests.timezone=America/Cayman
> -Dtests.file.encoding=UTF-8
> [junit4] FAILURE 0.64s J1 | TestBBoxStrategy.testOperations {#19
> seed=[631220E490F0DB53:EFFB6AF935D602D2]} <<<
> [junit4] > Throwable #1: java.lang.AssertionError: [BBoxWithin]
> Shouldn't match I#0:Rect(minX=180.0,maxX=180.0,minY=30.0,maxY=40.0)
> Q:Rect(minX=-180.0,maxX=-180.0,minY=-70.0,maxY=50.0)
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([631220E490F0DB53:EFFB6AF935D602D2]:0)
> [junit4] > at
> org.apache.lucene.spatial.prefix.RandomSpatialOpStrategyTestCase.fail(RandomSpatialOpStrategyTestCase.java:126)
> [junit4] > at
> org.apache.lucene.spatial.prefix.RandomSpatialOpStrategyTestCase.testOperation(RandomSpatialOpStrategyTestCase.java:115)
> [junit4] > at
> org.apache.lucene.spatial.prefix.RandomSpatialOpStrategyTestCase.testOperationRandomShapes(RandomSpatialOpStrategyTestCase.java:62)
> [junit4] > at
> org.apache.lucene.spatial.bbox.TestBBoxStrategy.testOperations(TestBBoxStrategy.java:109)
> [junit4] > at java.lang.Thread.run(Thread.java:745)
> [junit4] 2> أغس 16, 2014 9:16:18 ص
> org.apache.lucene.spatial.StrategyTestCase executeQueries
> [junit4] 2> INFO: testing queried for strategy BBoxStrategy
> field:bbox ctx=SpatialContext.GEO
> [junit4] 2> أغس 16, 2014 9:16:18 ص
> org.apache.lucene.spatial.StrategyTestCase executeQueries
> [junit4] 2> INFO: testing queried for strategy BBoxStrategy
> field:bbox ctx=SpatialContext.GEO
> [junit4] 2> NOTE: test params are: codec=CheapBastard,
> sim=RandomSimilarityProvider(queryNorm=false,coord=no): {bbox__xdl=DFR
> I(ne)LZ(0.3), bbox__maxX=DFR I(n)BZ(0.3), bbox__maxY=DFR I(F)3(800.0),
> bbox__minX=DFR I(ne)L2, bbox__minY=IB LL-D3(800.0)}, locale=ar_DZ,
> timezone=America/Cayman
> [junit4] 2> NOTE: Linux 3.13.0-32-generic amd64/Oracle Corporation
> 1.8.0_11 (64-bit)/cpus=8,threads=1,free=64697832,total=132120576
> [junit4] 2> NOTE: All tests run in this JVM: [SerializedStrategyTest,
> TestTermQueryPrefixGridStrategy, TestBBoxStrategy]
> [junit4] Completed on J1 in 10.24s, 30 tests, 1 failure, 1 skipped <<<
> FAILURES!
>
> [...truncated 35 lines...]
> BUILD FAILED
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:474: The
> following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:454: The
> following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The
> following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37:
> The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build.xml:544:
> The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:2106:
> The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/module-build.xml:60:
> The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1350:
> The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:965:
> There were test failures: 16 suites, 143 tests, 1 failure, 13 ignored (2
> assumptions)
>
> Total time: 42 minutes 43 seconds
> Build step 'Invoke Ant' marked build as failure
> [description-setter] Description set: Java: 64bit/jdk1.8.0_11
> -XX:+UseCompressedOops -XX:+UseG1GC
> Archiving artifacts
> Recording test results
> Email was triggered for: Failure - Any
> Sending email for trigger: Failure - Any
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.8.0_11) - Build #
11036 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/11036/
Java: 64bit/jdk1.8.0_11 -XX:+UseCompressedOops -XX:+UseG1GC
1 tests failed.
FAILED: org.apache.lucene.spatial.bbox.TestBBoxStrategy.testOperations {#19 seed=[631220E490F0DB53:EFFB6AF935D602D2]}
Error Message:
[BBoxWithin] Shouldn't match I#0:Rect(minX=180.0,maxX=180.0,minY=30.0,maxY=40.0) Q:Rect(minX=-180.0,maxX=-180.0,minY=-70.0,maxY=50.0)
Stack Trace:
java.lang.AssertionError: [BBoxWithin] Shouldn't match I#0:Rect(minX=180.0,maxX=180.0,minY=30.0,maxY=40.0) Q:Rect(minX=-180.0,maxX=-180.0,minY=-70.0,maxY=50.0)
at __randomizedtesting.SeedInfo.seed([631220E490F0DB53:EFFB6AF935D602D2]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.lucene.spatial.prefix.RandomSpatialOpStrategyTestCase.fail(RandomSpatialOpStrategyTestCase.java:126)
at org.apache.lucene.spatial.prefix.RandomSpatialOpStrategyTestCase.testOperation(RandomSpatialOpStrategyTestCase.java:115)
at org.apache.lucene.spatial.prefix.RandomSpatialOpStrategyTestCase.testOperationRandomShapes(RandomSpatialOpStrategyTestCase.java:62)
at org.apache.lucene.spatial.bbox.TestBBoxStrategy.testOperations(TestBBoxStrategy.java:109)
at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
Build Log:
[...truncated 9686 lines...]
[junit4] Suite: org.apache.lucene.spatial.bbox.TestBBoxStrategy
[junit4] 2> أغس 16, 2014 9:16:10 ص org.apache.lucene.spatial.StrategyTestCase executeQueries
[junit4] 2> INFO: testing queried for strategy BBoxStrategy field:bbox ctx=SpatialContext.GEO
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestBBoxStrategy -Dtests.method=testOperations -Dtests.seed=631220E490F0DB53 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ar_DZ -Dtests.timezone=America/Cayman -Dtests.file.encoding=UTF-8
[junit4] FAILURE 0.64s J1 | TestBBoxStrategy.testOperations {#19 seed=[631220E490F0DB53:EFFB6AF935D602D2]} <<<
[junit4] > Throwable #1: java.lang.AssertionError: [BBoxWithin] Shouldn't match I#0:Rect(minX=180.0,maxX=180.0,minY=30.0,maxY=40.0) Q:Rect(minX=-180.0,maxX=-180.0,minY=-70.0,maxY=50.0)
[junit4] > at __randomizedtesting.SeedInfo.seed([631220E490F0DB53:EFFB6AF935D602D2]:0)
[junit4] > at org.apache.lucene.spatial.prefix.RandomSpatialOpStrategyTestCase.fail(RandomSpatialOpStrategyTestCase.java:126)
[junit4] > at org.apache.lucene.spatial.prefix.RandomSpatialOpStrategyTestCase.testOperation(RandomSpatialOpStrategyTestCase.java:115)
[junit4] > at org.apache.lucene.spatial.prefix.RandomSpatialOpStrategyTestCase.testOperationRandomShapes(RandomSpatialOpStrategyTestCase.java:62)
[junit4] > at org.apache.lucene.spatial.bbox.TestBBoxStrategy.testOperations(TestBBoxStrategy.java:109)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[junit4] 2> أغس 16, 2014 9:16:18 ص org.apache.lucene.spatial.StrategyTestCase executeQueries
[junit4] 2> INFO: testing queried for strategy BBoxStrategy field:bbox ctx=SpatialContext.GEO
[junit4] 2> أغس 16, 2014 9:16:18 ص org.apache.lucene.spatial.StrategyTestCase executeQueries
[junit4] 2> INFO: testing queried for strategy BBoxStrategy field:bbox ctx=SpatialContext.GEO
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {bbox__xdl=DFR I(ne)LZ(0.3), bbox__maxX=DFR I(n)BZ(0.3), bbox__maxY=DFR I(F)3(800.0), bbox__minX=DFR I(ne)L2, bbox__minY=IB LL-D3(800.0)}, locale=ar_DZ, timezone=America/Cayman
[junit4] 2> NOTE: Linux 3.13.0-32-generic amd64/Oracle Corporation 1.8.0_11 (64-bit)/cpus=8,threads=1,free=64697832,total=132120576
[junit4] 2> NOTE: All tests run in this JVM: [SerializedStrategyTest, TestTermQueryPrefixGridStrategy, TestBBoxStrategy]
[junit4] Completed on J1 in 10.24s, 30 tests, 1 failure, 1 skipped <<< FAILURES!
[...truncated 35 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:474: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:454: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build.xml:544: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:2106: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/module-build.xml:60: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1350: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:965: There were test failures: 16 suites, 143 tests, 1 failure, 13 ignored (2 assumptions)
Total time: 42 minutes 43 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 64bit/jdk1.8.0_11 -XX:+UseCompressedOops -XX:+UseG1GC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-trunk-Linux (64bit/jdk1.7.0_65) - Build #
11035 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/11035/
Java: 64bit/jdk1.7.0_65 -XX:-UseCompressedOops -XX:+UseParallelGC
1 tests failed.
REGRESSION: org.apache.solr.cloud.OverseerTest.testOverseerFailure
Error Message:
Could not register as the leader because creating the ephemeral registration node in ZooKeeper failed
Stack Trace:
org.apache.solr.common.SolrException: Could not register as the leader because creating the ephemeral registration node in ZooKeeper failed
at __randomizedtesting.SeedInfo.seed([AAD62B92F4F41B85:AEDEA461E651F4A4]:0)
at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:150)
at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:163)
at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:125)
at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:155)
at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:314)
at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:221)
at org.apache.solr.cloud.OverseerTest$MockZKController.publishState(OverseerTest.java:157)
at org.apache.solr.cloud.OverseerTest.testOverseerFailure(OverseerTest.java:662)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /collections/collection1/leaders/shard1
at org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
at org.apache.solr.common.cloud.SolrZkClient$11.execute(SolrZkClient.java:462)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:459)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:416)
at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:403)
at org.apache.solr.cloud.ShardLeaderElectionContextBase$1.execute(ElectionContext.java:142)
at org.apache.solr.common.util.RetryUtil.retryOnThrowable(RetryUtil.java:31)
at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:137)
... 47 more
Build Log:
[...truncated 11496 lines...]
[junit4] Suite: org.apache.solr.cloud.OverseerTest
[junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.OverseerTest-AAD62B92F4F41B85-001/init-core-data-001
[junit4] 2> 1801132 T5056 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false)
[junit4] 2> 1801132 T5056 oas.SolrTestCaseJ4.initCore ####initCore
[junit4] 2> 1801132 T5056 oas.SolrTestCaseJ4.initCore ####initCore end
[junit4] 2> 1801136 T5056 oas.SolrTestCaseJ4.setUp ###Starting testReplay
[junit4] 2> 1801136 T5056 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1801137 T5057 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 1801237 T5056 oasc.ZkTestServer.run start zk server on port:41938
[junit4] 2> 1801238 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1801240 T5063 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@33622d04 name:ZooKeeperConnection Watcher:127.0.0.1:41938/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1801240 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1801241 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1801242 T5065 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5fd2b791 name:ZooKeeperConnection Watcher:127.0.0.1:41938 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1801242 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1801244 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1801245 T5067 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@76b42a79 name:ZooKeeperConnection Watcher:127.0.0.1:41938 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1801245 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1801245 T5056 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 1801249 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 1801250 T5056 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 1801251 T5056 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 1801252 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1801254 T5056 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 1801263 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1801265 T5069 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3b1d8078 name:ZooKeeperConnection Watcher:127.0.0.1:41938/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1801265 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1801265 T5056 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: retry=false
[junit4] 2> 1801342 T5056 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 1801344 T5056 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 1801346 T5056 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1801347 T5056 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:41938_solr
[junit4] 2> 1801347 T5056 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 1801348 T5056 oasc.Overseer.start Overseer (id=92286963279986691-127.0.0.1:41938_solr-n_0000000000) starting
[junit4] 2> 1801350 T5056 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 1801352 T5056 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 1801353 T5056 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 1801355 T5056 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 1801357 T5056 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 1801360 T5056 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 1801360 T5071 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 1801362 T5070 oasc.Overseer$ClusterStateUpdater.run Replaying operations from work queue.
[junit4] 2> 1801362 T5070 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "shard":"s1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core1",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"}
[junit4] 2> 1801363 T5073 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 1801365 T5070 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "shard":"s1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core2",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"}
[junit4] 2> 1801366 T5073 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 1801366 T5070 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 1801368 T5070 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "shard":"s1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core3",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"}
[junit4] 2> 1801369 T5074 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1801370 T5073 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 1801464 T5056 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:41938 41938
[junit4] 2> 1802006 T5057 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:41938 41938
[junit4] 2> 1802007 T5056 oas.SolrTestCaseJ4.tearDown ###Ending testReplay
[junit4] 2> 1802008 T5056 oasc.Overseer.close Overseer (id=92286963279986691-127.0.0.1:41938_solr-n_0000000000) closing
[junit4] 2> 1802008 T5070 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:41938_solr
[junit4] 2> 1802012 T5075 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:321)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:89)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:293)
[junit4] 2>
[junit4] 2> 1802016 T5056 oas.SolrTestCaseJ4.setUp ###Starting testStateChange
[junit4] 2> 1802017 T5056 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1802017 T5076 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 1802117 T5056 oasc.ZkTestServer.run start zk server on port:51921
[junit4] 2> 1802118 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1802120 T5082 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@41b7ad2c name:ZooKeeperConnection Watcher:127.0.0.1:51921/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1802120 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1802121 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1802122 T5084 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4490bec6 name:ZooKeeperConnection Watcher:127.0.0.1:51921 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1802122 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1802125 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1802125 T5086 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4d6dcc1a name:ZooKeeperConnection Watcher:127.0.0.1:51921 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1802126 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1802126 T5056 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 1802128 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 1802130 T5056 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 1802131 T5056 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 1802131 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1802133 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1802134 T5088 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5946fa2f name:ZooKeeperConnection Watcher:127.0.0.1:51921/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1802134 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1802134 T5056 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: retry=false
[junit4] 2> 1802136 T5056 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 1802137 T5056 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 1802138 T5056 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1802139 T5056 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:51921_solr
[junit4] 2> 1802139 T5056 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 1802140 T5056 oasc.Overseer.start Overseer (id=92286963340279811-127.0.0.1:51921_solr-n_0000000000) starting
[junit4] 2> 1802142 T5056 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 1802143 T5056 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 1802145 T5056 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 1802146 T5056 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 1802148 T5056 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 1802150 T5056 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 1802152 T5056 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 1802153 T5090 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 1802154 T5089 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 1802155 T5089 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core1",
[junit4] 2> "roles":"",
[junit4] 2> "state":"recovering"}
[junit4] 2> 1802155 T5089 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 1802156 T5092 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1802157 T5093 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 1802213 T5092 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1802214 T5089 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4] 2> "operation":"state",
[junit4] 2> "base_url":"http://127.0.0.1/solr",
[junit4] 2> "node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "core":"core1",
[junit4] 2> "roles":"",
[junit4] 2> "state":"active"}
[junit4] 2> 1802214 T5089 oasc.Overseer$ClusterStateUpdater.updateState node=core_node1 is already registered
[junit4] 2> 1802214 T5089 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4] 2> 1802317 T5093 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 0)
[junit4] 2> 1802365 T5056 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:51921 51921
[junit4] 2> 1803051 T5076 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:51921 51921
[junit4] 2> 1803052 T5056 oas.SolrTestCaseJ4.tearDown ###Ending testStateChange
[junit4] 2> 1803052 T5056 oasc.Overseer.close Overseer (id=92286963340279811-127.0.0.1:51921_solr-n_0000000000) closing
[junit4] 2> 1803053 T5089 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:51921_solr
[junit4] 2> 1803054 T5094 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:321)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:89)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:293)
[junit4] 2>
[junit4] 2> 1803061 T5056 oas.SolrTestCaseJ4.setUp ###Starting testShardAssignment
[junit4] 2> 1803062 T5056 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1803062 T5095 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 1803162 T5056 oasc.ZkTestServer.run start zk server on port:48207
[junit4] 2> 1803163 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1803166 T5101 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@356cb26a name:ZooKeeperConnection Watcher:127.0.0.1:48207 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1803166 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1803169 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1803170 T5103 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@11fffbac name:ZooKeeperConnection Watcher:127.0.0.1:48207 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1803170 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1803171 T5056 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 1803173 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1803174 T5105 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@411ab9ec name:ZooKeeperConnection Watcher:127.0.0.1:48207/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1803175 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1803175 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 1803176 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1803177 T5107 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@615ea275 name:ZooKeeperConnection Watcher:127.0.0.1:48207/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1803177 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1803177 T5056 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: retry=false
[junit4] 2> 1803179 T5056 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 1803180 T5056 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 1803182 T5056 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1803183 T5056 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:48207_solr
[junit4] 2> 1803183 T5056 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 1803184 T5056 oasc.Overseer.start Overseer (id=92286963404570627-127.0.0.1:48207_solr-n_0000000000) starting
[junit4] 2> 1803185 T5056 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 1803187 T5056 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 1803188 T5056 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 1803189 T5056 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 1803191 T5056 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 1803193 T5056 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 1803195 T5056 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 1803196 T5109 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 1803196 T5056 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 1803196 T5108 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 1803197 T5056 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 1803198 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1803199 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1803200 T5112 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6a746293 name:ZooKeeperConnection Watcher:127.0.0.1:48207/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1803201 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1803201 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1803203 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1
[junit4] 2> 1803214 T5115 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1803215 T5108 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 1803215 T5108 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2, shard3]
[junit4] 2> 1803216 T5108 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
[junit4] 2> 1803218 T5114 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1803218 T5113 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1803714 T5056 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
[junit4] 2> 1803719 T5056 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard3
[junit4] 2> 1803722 T5115 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1803724 T5108 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core2",
[junit4] 2> "core_node_name":"node2",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 1803724 T5108 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 1803724 T5108 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 1803725 T5115 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1803726 T5114 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1803726 T5113 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1804223 T5056 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 1804228 T5056 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4] 2> 1804230 T5115 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1804233 T5108 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core3",
[junit4] 2> "core_node_name":"node3",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 1804233 T5108 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 1804233 T5108 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 1804235 T5115 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1804235 T5114 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1804235 T5113 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1804731 T5056 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1804736 T5056 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4] 2> 1804738 T5115 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1804741 T5108 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core4",
[junit4] 2> "core_node_name":"node4",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 1804741 T5108 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 1804741 T5108 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
[junit4] 2> 1804742 T5115 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1804743 T5114 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1804743 T5113 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1805239 T5056 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
[junit4] 2> 1805246 T5115 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1805247 T5108 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core5",
[junit4] 2> "core_node_name":"node5",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 1805247 T5108 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 1805248 T5108 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4] 2> 1805350 T5114 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1805350 T5113 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1805746 T5056 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4] 2> 1805750 T5115 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1805752 T5108 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"active",
[junit4] 2> "node_name":"127.0.0.1",
[junit4] 2> "core":"core6",
[junit4] 2> "core_node_name":"node6",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"3",
[junit4] 2> "base_url":"http://127.0.0.1/solr/"}
[junit4] 2> 1805752 T5108 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4] 2> 1805752 T5108 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4] 2> 1805855 T5114 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1805855 T5113 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1806250 T5056 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4] 2> 1806257 T5112 oaz.ClientCnxn$EventThread.processEvent ERROR Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@41ee8656 rejected from java.util.concurrent.ThreadPoolExecutor@12b599eb[Shutting down, pool size = 2, active threads = 2, queued tasks = 0, completed tasks = 8]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
[junit4] 2> at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:110)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3.process(SolrZkClient.java:206)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
[junit4] 2>
[junit4] 2> 1806257 T5116 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard2/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:290)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:287)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:287)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:358)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:209)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 1807757 T5113 oasc.LeaderElector$ElectionWatcher.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard3/election
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:290)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:287)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:287)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:94)
[junit4] 2> at org.apache.solr.cloud.LeaderElector.access$200(LeaderElector.java:55)
[junit4] 2> at org.apache.solr.cloud.LeaderElector$ElectionWatcher.process(LeaderElector.java:358)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:209)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4] 2> at java.lang.Thread.run(Thread.java:745)
[junit4] 2>
[junit4] 2> 1807920 T5056 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:48207 48207
[junit4] 2> 1808536 T5095 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:48207 48207
[junit4] 2> 1808537 T5056 oas.SolrTestCaseJ4.tearDown ###Ending testShardAssignment
[junit4] 2> 1808537 T5056 oasc.Overseer.close Overseer (id=92286963404570627-127.0.0.1:48207_solr-n_0000000000) closing
[junit4] 2> 1808537 T5108 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:48207_solr
[junit4] 2> 1808539 T5118 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:321)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:89)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:293)
[junit4] 2>
[junit4] 2> 1808547 T5056 oas.SolrTestCaseJ4.setUp ###Starting testShardAssignmentBigger
[junit4] 2> 1808548 T5056 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1808548 T5119 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4] 2> 1808648 T5056 oasc.ZkTestServer.run start zk server on port:59640
[junit4] 2> 1808649 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1808651 T5125 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5728fdf3 name:ZooKeeperConnection Watcher:127.0.0.1:59640 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1808651 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1808654 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1808655 T5127 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e6e2473 name:ZooKeeperConnection Watcher:127.0.0.1:59640 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1808655 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1808655 T5056 oascc.SolrZkClient.makePath makePath: /solr
[junit4] 2> 1808659 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1808660 T5129 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a90b931 name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1808660 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1808660 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4] 2> 1808661 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1808662 T5131 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@381b9bc9 name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1808663 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1808663 T5056 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: retry=false
[junit4] 2> 1808667 T5056 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 1808668 T5056 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 1808670 T5056 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1808671 T5056 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:59640_solr
[junit4] 2> 1808672 T5056 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 1808673 T5056 oasc.Overseer.start Overseer (id=92286963764166659-127.0.0.1:59640_solr-n_0000000000) starting
[junit4] 2> 1808674 T5056 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 1808676 T5056 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 1808678 T5056 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 1808680 T5056 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 1808682 T5056 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 1808684 T5056 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 1808687 T5056 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 1808688 T5133 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 1808689 T5132 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 1808689 T5056 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4] 2> 1808691 T5056 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4] 2> 1808692 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1808694 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1808695 T5136 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@16adf439 name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1808696 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1808696 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1808698 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node0
[junit4] 2> 1808700 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1808702 T5140 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@325f6509 name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1808702 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1808703 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1808704 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
[junit4] 2> 1808706 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1808707 T5143 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5e952427 name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1808707 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1808708 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1808709 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node2
[junit4] 2> 1808711 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1808712 T5146 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2a459104 name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1808712 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1808713 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1808714 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node3
[junit4] 2> 1808716 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1808718 T5149 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2e454ecf name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1808718 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1808719 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1808720 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node4
[junit4] 2> 1808723 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1808725 T5152 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@78a9c074 name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1808725 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1808730 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1808732 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node5
[junit4] 2> 1808741 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1808742 T5155 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1d423e21 name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1808742 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1808743 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1808744 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node6
[junit4] 2> 1808747 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1808748 T5158 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@132e065c name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1808748 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1808749 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1808750 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node7
[junit4] 2> 1808754 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1808755 T5161 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4bb0e9ff name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1808755 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1808756 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1808757 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node8
[junit4] 2> 1808761 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1808762 T5164 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@12c4a471 name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1808763 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1808764 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1808765 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node9
[junit4] 2> 1808769 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1808770 T5167 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@26ec1a73 name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1808770 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1808771 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1808773 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node10
[junit4] 2> 1808779 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1808780 T5170 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@36735a74 name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1808780 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1808781 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1808783 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node11
[junit4] 2> 1808791 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1808792 T5173 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@49e8a96d name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1808793 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1808793 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1808795 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node12
[junit4] 2> 1808804 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1808807 T5176 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5c3e9116 name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1808807 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1808808 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1808809 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node13
[junit4] 2> 1808813 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1808821 T5179 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@414b3086 name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1808821 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1808822 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1808823 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node14
[junit4] 2> 1808827 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1808828 T5182 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@276c4421 name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1808828 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1808828 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1808830 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node15
[junit4] 2> 1808833 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1808834 T5185 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7998b3dc name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1808834 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1808835 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1808836 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node16
[junit4] 2> 1808841 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1808842 T5188 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2cefc85f name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1808843 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1808843 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1808845 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node17
[junit4] 2> 1808856 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1808858 T5191 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2f8fc03c name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1808859 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1808859 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1808861 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node18
[junit4] 2> 1808864 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1808865 T5194 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@577f52ed name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1808866 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1808866 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1808867 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node19
[junit4] 2> 1808872 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1808873 T5197 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4beb1faf name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1808873 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1808873 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1808874 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node20
[junit4] 2> 1808878 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1808879 T5200 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2e846fc8 name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1808879 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1808880 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1808881 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node21
[junit4] 2> 1808885 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1808886 T5203 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d737d0f name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1808886 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1808887 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1808888 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node22
[junit4] 2> 1808892 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1808892 T5206 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@62969d82 name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1808893 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1808893 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1808894 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node23
[junit4] 2> 1808898 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1808899 T5209 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7b0d0647 name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1808899 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1808900 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1808901 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node24
[junit4] 2> 1808905 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1808905 T5212 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@26592f27 name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1808906 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1808906 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1808907 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node25
[junit4] 2> 1808918 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1808919 T5215 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@515e58a1 name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1808919 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1808920 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1808921 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node26
[junit4] 2> 1808926 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1808927 T5218 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@9071edf name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1808928 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1808928 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1808929 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node27
[junit4] 2> 1808933 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1808934 T5221 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2b9d0089 name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1808934 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1808935 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1808936 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node28
[junit4] 2> 1808940 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1808941 T5224 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@61181327 name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1808941 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1808942 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1808943 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node29
[junit4] 2> 1808959 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1808960 T5227 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@57719a55 name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1808961 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1808961 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1808963 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node30
[junit4] 2> 1808968 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1808969 T5230 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@22d9f49b name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1808969 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1808969 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1808971 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node31
[junit4] 2> 1808975 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1808976 T5233 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4ccc24bb name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1808976 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1808977 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1808978 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node32
[junit4] 2> 1808983 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1808984 T5236 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4188dfe4 name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1808984 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1808984 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1808985 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node33
[junit4] 2> 1808989 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1808990 T5239 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@40042304 name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1808990 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1808991 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1808992 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node34
[junit4] 2> 1809001 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1809002 T5242 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@16862c1a name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1809002 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1809003 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1809004 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node35
[junit4] 2> 1809007 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1809009 T5245 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@41ab46b3 name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1809009 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1809009 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1809011 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node36
[junit4] 2> 1809015 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1809017 T5248 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2d0132fa name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1809017 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1809017 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1809019 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node37
[junit4] 2> 1809023 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1809024 T5251 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7e7b82ca name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1809024 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1809025 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1809026 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node38
[junit4] 2> 1809031 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1809032 T5254 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@680a376a name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1809032 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1809033 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1809034 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node39
[junit4] 2> 1809087 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1809088 T5257 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6c14be6b name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1809088 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1809089 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1809090 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node40
[junit4] 2> 1809098 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1809099 T5260 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4066a62 name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1809100 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1809100 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1809102 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node41
[junit4] 2> 1809111 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1809112 T5263 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3ceaf7dc name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1809112 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1809113 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1809114 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node42
[junit4] 2> 1809121 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1809122 T5266 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@90a3f29 name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1809122 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1809123 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1809125 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node43
[junit4] 2> 1809130 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1809131 T5269 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@67d12d90 name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1809131 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1809132 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1809133 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node44
[junit4] 2> 1809139 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1809139 T5272 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@156eb1cb name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1809140 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1809141 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1809142 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node45
[junit4] 2> 1809147 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1809148 T5275 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2dd1c76c name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1809149 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1809149 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1809150 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node46
[junit4] 2> 1809516 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1809516 T5278 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7f4e3b3b name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1809517 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1809518 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1809519 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node47
[junit4] 2> 1809532 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1809534 T5281 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@169560b3 name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1809534 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1809535 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1809537 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node48
[junit4] 2> 1809595 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1809595 T5284 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2e1d9dce name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1809596 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1809596 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1809597 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node49
[junit4] 2> 1809648 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1809650 T5287 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3f4fa4ae name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1809650 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1809651 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1809652 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node50
[junit4] 2> 1809662 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1809663 T5290 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@524fb71 name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1809663 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1809664 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1809666 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node51
[junit4] 2> 1809673 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1809674 T5293 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5d201898 name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1809674 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1809674 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1809676 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node52
[junit4] 2> 1809683 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1809684 T5296 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@36930836 name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1809685 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1809685 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1809686 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node53
[junit4] 2> 1809693 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1809694 T5299 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@33d490d3 name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1809694 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1809695 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1809696 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node54
[junit4] 2> 1809702 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1809706 T5302 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6a73bb0b name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1809706 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1809708 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1809709 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node55
[junit4] 2> 1809719 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1809720 T5305 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2d7a951 name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1809720 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1809721 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1809725 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node56
[junit4] 2> 1809736 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1809736 T5308 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7d5be29c name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4] 2> 1809736 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1809737 T5056 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4] 2> 1809738 T5056 oascc.SolrZkClient.makePath makePath: /live_nodes/node57
[junit4] 2> 1809747 T5056 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4] 2> 1809748 T5311 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2e2d1237 name:ZooKeeperConnection Watcher:127.0.0.1:59640/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[j
[...truncated too long message...]
ncConnected type:None path:null path:null type:None
[junit4] 2> 1890128 T5056 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4] 2> 1890128 T5056 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: retry=false
[junit4] 2> 1890130 T5056 oascc.SolrZkClient.makePath makePath: /overseer_elect
[junit4] 2> 1890131 T5056 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4] 2> 1890133 T5056 oasc.Overseer.close Overseer (id=null) closing
[junit4] 2> 1890134 T5056 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:33490_solr
[junit4] 2> 1890135 T5056 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4] 2> 1890136 T5056 oasc.Overseer.start Overseer (id=92286969101615108-127.0.0.1:33490_solr-n_0000000000) starting
[junit4] 2> 1890138 T5056 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4] 2> 1890140 T5056 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4] 2> 1890141 T5056 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
[junit4] 2> 1890143 T5056 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
[junit4] 2> 1890145 T5056 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
[junit4] 2> 1890146 T5056 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4] 2> 1890149 T5056 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
[junit4] 2> 1890150 T5806 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4] 2> 1890150 T5805 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4] 2> 1890152 T5805 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=12 message={
[junit4] 2> "operation":"state",
[junit4] 2> "state":"recovering",
[junit4] 2> "node_name":"node1",
[junit4] 2> "core":"core1",
[junit4] 2> "core_node_name":"node1",
[junit4] 2> "collection":"collection1",
[junit4] 2> "numShards":"12",
[junit4] 2> "base_url":"http://node1/solr/"}
[junit4] 2> 1890152 T5805 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2, shard3, shard4, shard5, shard6, shard7, shard8, shard9, shard10, shard11, shard12]
[junit4] 2> 1890152 T5805 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard10
[junit4] 2> 1890154 T5808 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1890155 T5801 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1890155 T5802 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4] 2> 1890651 T5056 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard10/election
[junit4] 2> 1890655 T5056 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard10
[junit4] 2> 1890658 T5808 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
[junit4] 2> 1890671 T5056 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:33490 33490
[junit4] 2> 1891218 T5788 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:33490 33490
[junit4] 2> 1891229 T5056 oas.SolrTestCaseJ4.tearDown ###Ending testPlaceholders
[junit4] 2> 1891229 T5056 oasc.Overseer.close Overseer (id=92286969101615108-127.0.0.1:33490_solr-n_0000000000) closing
[junit4] 2> 1891229 T5805 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:33490_solr
[junit4] 2> 1891230 T5809 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:307)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$8.execute(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:304)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:321)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:89)
[junit4] 2> at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:293)
[junit4] 2>
[junit4] 2> 1894231 T5056 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./temp/solr.cloud.OverseerTest-AAD62B92F4F41B85-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene410), sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=ga_IE, timezone=America/Rankin_Inlet
[junit4] 2> NOTE: Linux 3.13.0-32-generic amd64/Oracle Corporation 1.7.0_65 (64-bit)/cpus=8,threads=1,free=237859336,total=393740288
[junit4] 2> NOTE: All tests run in this JVM: [DocValuesTest, TestMergePolicyConfig, SchemaVersionSpecificBehaviorTest, TestSearcherReuse, DistributedDebugComponentTest, FullSolrCloudDistribCmdsTest, TestManagedStopFilterFactory, TestCollectionAPI, DateMathParserTest, TestAnalyzeInfixSuggestions, SpatialFilterTest, ConvertedLegacyTest, TestIndexSearcher, URLClassifyProcessorTest, SortByFunctionTest, TestPostingsSolrHighlighter, TestFileDictionaryLookup, CurrencyFieldXmlFileTest, ShardSplitTest, TestInitQParser, TestCloudInspectUtil, TestOmitPositions, QueryParsingTest, SliceStateTest, TestCollationFieldDocValues, AlternateDirectoryTest, TestSweetSpotSimilarityFactory, SearchHandlerTest, FileBasedSpellCheckerTest, TestUniqueKeyFieldResource, TestSolrQueryParserResource, TestPhraseSuggestions, TestDocBasedVersionConstraints, SolrXmlInZkTest, DistribDocExpirationUpdateProcessorTest, TestClassNameShortening, TestUtils, TestIndexingPerformance, TestFunctionQuery, TestSolrXml, TestInfoStreamLogging, HighlighterMaxOffsetTest, AbstractAnalyticsFacetTest, SolrInfoMBeanTest, TermVectorComponentDistributedTest, CollectionsAPIDistributedZkTest, ChaosMonkeySafeLeaderTest, SolrPluginUtilsTest, TestJmxMonitoredMap, TestRequestStatusCollectionAPI, TestCollationField, HardAutoCommitTest, TestQueryUtils, OpenCloseCoreStressTest, AssignTest, TestLazyCores, ScriptEngineTest, TestRecoveryHdfs, TestPivotHelperCode, TestLeaderElectionZkExpiry, DateFieldTest, TestFieldSortValues, TestElisionMultitermQuery, DebugComponentTest, ClusterStateTest, AnalysisErrorHandlingTest, SynonymTokenizerTest, PreAnalyzedFieldTest, TestAtomicUpdateErrorCases, TestUpdate, TestCSVResponseWriter, TestValueSourceCache, TestNonNRTOpen, CustomCollectionTest, CopyFieldTest, TestPHPSerializedResponseWriter, SolrIndexConfigTest, TestXIncludeConfig, TestFieldTypeCollectionResource, MultiTermTest, RemoteQueryErrorTest, TestReRankQParserPlugin, CoreAdminRequestStatusTest, ExpressionTest, TestCharFilters, TestDistribDocBasedVersion, TestStandardQParsers, TestImplicitCoreProperties, TestIntervalFaceting, HdfsSyncSliceTest, CursorMarkTest, TestHighlightDedupGrouping, TestDynamicFieldCollectionResource, TestBM25SimilarityFactory, DistanceFunctionTest, DistributedTermsComponentTest, DistributedFacetPivotLargeTest, TestRandomFaceting, TestNoOpRegenerator, TestManagedResource, FunctionTest, BasicZkTest, BasicDistributedZk2Test, UnloadDistributedZkTest, OverseerTest]
[junit4] Completed on J1 in 93.17s, 9 tests, 1 error <<< FAILURES!
[...truncated 671 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:474: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:454: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:189: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:496: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1350: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:965: There were test failures: 430 suites, 1782 tests, 1 error, 40 ignored (19 assumptions)
Total time: 92 minutes 41 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 64bit/jdk1.7.0_65 -XX:-UseCompressedOops -XX:+UseParallelGC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any