You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2013/03/02 22:57:44 UTC
[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0-ea-b78) - Build #
4542 - Still Failing!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/4542/
Java: 32bit/jdk1.8.0-ea-b78 -client -XX:+UseConcMarkSweepGC
2 tests failed.
REGRESSION: org.apache.solr.cloud.BasicDistributedZk2Test.testDistribSearch
Error Message:
expected:<1> but was:<0>
Stack Trace:
java.lang.AssertionError: expected:<1> but was:<0>
at __randomizedtesting.SeedInfo.seed([F7EF1912CDA91723:7609970ABAF6771F]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.BasicDistributedZk2Test.testNodeWithoutCollectionForwarding(BasicDistributedZk2Test.java:209)
at org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:89)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:487)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
REGRESSION: org.apache.solr.cloud.UnloadDistributedZkTest.testDistribSearch
Error Message:
Still found shard
Stack Trace:
java.lang.AssertionError: Still found shard
at __randomizedtesting.SeedInfo.seed([F7EF1912CDA91723:7609970ABAF6771F]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.UnloadDistributedZkTest.testUnloadShardAndCollection(UnloadDistributedZkTest.java:124)
at org.apache.solr.cloud.UnloadDistributedZkTest.doTest(UnloadDistributedZkTest.java:77)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:487)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Build Log:
[...truncated 9067 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZk2Test
[junit4:junit4] 2> 0 T558 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4] 2> 2 T558 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362260776097
[junit4:junit4] 2> 3 T558 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 3 T559 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 4 T559 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4] 2> 4 T559 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4] 2> 4 T559 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4] 2> 5 T559 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4] 2> 5 T559 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4] 2> 103 T558 oasc.ZkTestServer.run start zk server on port:54542
[junit4:junit4] 2> 104 T558 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54542 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@1be1abf
[junit4:junit4] 2> 105 T558 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 105 T564 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54542. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 106 T564 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54542, initiating session
[junit4:junit4] 2> 106 T560 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:42437
[junit4:junit4] 2> 106 T560 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:42437
[junit4:junit4] 2> 107 T562 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4] 2> 207 T562 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d2d11d4a40000 with negotiated timeout 10000 for client /127.0.0.1:42437
[junit4:junit4] 2> 207 T564 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54542, sessionid = 0x13d2d11d4a40000, negotiated timeout = 10000
[junit4:junit4] 2> 207 T565 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1be1abf name:ZooKeeperConnection Watcher:127.0.0.1:54542 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 208 T558 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 208 T558 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 211 T563 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d2d11d4a40000
[junit4:junit4] 2> 212 T565 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 212 T560 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:42437 which had sessionid 0x13d2d11d4a40000
[junit4:junit4] 2> 212 T558 oaz.ZooKeeper.close Session: 0x13d2d11d4a40000 closed
[junit4:junit4] 2> 213 T558 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54542/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@179e478
[junit4:junit4] 2> 214 T558 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 214 T566 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54542. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 214 T566 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54542, initiating session
[junit4:junit4] 2> 214 T560 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:42438
[junit4:junit4] 2> 215 T560 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:42438
[junit4:junit4] 2> 215 T562 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d2d11d4a40001 with negotiated timeout 10000 for client /127.0.0.1:42438
[junit4:junit4] 2> 215 T566 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54542, sessionid = 0x13d2d11d4a40001, negotiated timeout = 10000
[junit4:junit4] 2> 216 T567 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@179e478 name:ZooKeeperConnection Watcher:127.0.0.1:54542/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 216 T558 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 216 T558 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 219 T558 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 224 T558 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 226 T558 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 227 T558 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 228 T558 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 231 T558 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 232 T558 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 235 T558 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 235 T558 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 237 T558 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 238 T558 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 240 T558 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 241 T558 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 243 T558 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 244 T558 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 246 T558 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 246 T558 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 248 T558 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 249 T558 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 251 T558 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 252 T558 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 254 T563 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d2d11d4a40001
[junit4:junit4] 2> 254 T567 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 254 T560 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:42438 which had sessionid 0x13d2d11d4a40001
[junit4:junit4] 2> 254 T558 oaz.ZooKeeper.close Session: 0x13d2d11d4a40001 closed
[junit4:junit4] 2> 346 T558 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 352 T558 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:46561
[junit4:junit4] 2> 352 T558 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 353 T558 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 353 T558 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362260776349
[junit4:junit4] 2> 354 T558 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362260776349/solr.xml
[junit4:junit4] 2> 354 T558 oasc.CoreContainer.<init> New CoreContainer 11847213
[junit4:junit4] 2> 355 T558 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362260776349/'
[junit4:junit4] 2> 356 T558 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362260776349/'
[junit4:junit4] 2> 389 T558 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 390 T558 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 390 T558 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 391 T558 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 391 T558 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 392 T558 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 392 T558 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 393 T558 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 393 T558 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 394 T558 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 408 T558 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 425 T558 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:54542/solr
[junit4:junit4] 2> 426 T558 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 426 T558 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54542 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@1bffbd7
[junit4:junit4] 2> 427 T577 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54542. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 427 T558 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 428 T560 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:42441
[junit4:junit4] 2> 428 T577 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54542, initiating session
[junit4:junit4] 2> 428 T560 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:42441
[junit4:junit4] 2> 430 T562 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d2d11d4a40002 with negotiated timeout 20000 for client /127.0.0.1:42441
[junit4:junit4] 2> 430 T577 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54542, sessionid = 0x13d2d11d4a40002, negotiated timeout = 20000
[junit4:junit4] 2> 430 T578 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1bffbd7 name:ZooKeeperConnection Watcher:127.0.0.1:54542 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 430 T558 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 431 T563 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d2d11d4a40002
[junit4:junit4] 2> 432 T560 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:42441 which had sessionid 0x13d2d11d4a40002
[junit4:junit4] 2> 432 T578 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 432 T558 oaz.ZooKeeper.close Session: 0x13d2d11d4a40002 closed
[junit4:junit4] 2> 433 T558 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 440 T558 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54542/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@e330d8
[junit4:junit4] 2> 441 T579 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54542. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 441 T558 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 441 T579 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54542, initiating session
[junit4:junit4] 2> 441 T560 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:42442
[junit4:junit4] 2> 442 T560 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:42442
[junit4:junit4] 2> 443 T562 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d2d11d4a40003 with negotiated timeout 20000 for client /127.0.0.1:42442
[junit4:junit4] 2> 443 T579 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54542, sessionid = 0x13d2d11d4a40003, negotiated timeout = 20000
[junit4:junit4] 2> 443 T580 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e330d8 name:ZooKeeperConnection Watcher:127.0.0.1:54542/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 443 T558 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 445 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a40003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 446 T558 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 447 T558 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46561_
[junit4:junit4] 2> 448 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a40003 type:delete cxid:0x8 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:46561_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:46561_
[junit4:junit4] 2> 449 T558 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:46561_
[junit4:junit4] 2> 451 T558 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 454 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a40003 type:delete cxid:0x16 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4] 2> 454 T558 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 457 T558 oasc.Overseer.start Overseer (id=89277122222489603-127.0.0.1:46561_-n_0000000000) starting
[junit4:junit4] 2> 458 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a40003 type:create cxid:0x1b zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 459 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a40003 type:create cxid:0x1c zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 459 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a40003 type:create cxid:0x1d zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 460 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a40003 type:create cxid:0x1e zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 461 T582 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 466 T558 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 468 T558 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 470 T581 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 472 T583 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362260776349/collection1
[junit4:junit4] 2> 472 T583 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 473 T583 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 473 T583 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 474 T583 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362260776349/collection1/'
[junit4:junit4] 2> 474 T583 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362260776349/collection1/lib/README' to classloader
[junit4:junit4] 2> 475 T583 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362260776349/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 519 T583 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 579 T583 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 580 T583 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 585 T583 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 1296 T583 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 1309 T583 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1312 T583 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1320 T583 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1327 T583 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1330 T583 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1330 T583 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1331 T583 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 1332 T583 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1332 T583 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1332 T583 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 1333 T583 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362260776349/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/control/data/
[junit4:junit4] 2> 1333 T583 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@e089a3
[junit4:junit4] 2> 1333 T583 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1334 T583 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/control/data forceNew:false
[junit4:junit4] 2> 1334 T583 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/control/data
[junit4:junit4] 2> 1334 T583 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/control/data/index/
[junit4:junit4] 2> 1334 T583 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1334 T583 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/control/data/index forceNew:false
[junit4:junit4] 2> 1336 T583 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@10fa463 lockFactory=org.apache.lucene.store.NativeFSLockFactory@57e1e6),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 1336 T583 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 1336 T583 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/control/data/index
[junit4:junit4] 2> 1337 T583 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1338 T583 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1338 T583 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1338 T583 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1339 T583 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1339 T583 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1339 T583 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1339 T583 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1340 T583 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1342 T583 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1344 T583 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/control/data
[junit4:junit4] 2> 1344 T583 oass.SolrIndexSearcher.<init> Opening Searcher@1e5f22c main
[junit4:junit4] 2> 1345 T583 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/control/data/tlog
[junit4:junit4] 2> 1345 T583 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1345 T583 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1348 T583 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/control/data
[junit4:junit4] 2> 1348 T584 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1e5f22c main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1355 T583 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1355 T583 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1356 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a40003 type:create cxid:0x50 zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4] 2> 1973 T581 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1974 T581 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "base_url":"http://127.0.0.1:46561",
[junit4:junit4] 2> "node_name":"127.0.0.1:46561_",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null}
[junit4:junit4] 2> 1974 T581 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 1974 T581 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 1975 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a40003 type:create cxid:0x58 zxid:0x2c txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4] 2> 1978 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 2357 T583 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 2357 T583 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:46561 collection:control_collection shard:shard1
[junit4:junit4] 2> 2358 T583 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 2362 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a40003 type:delete cxid:0x72 zxid:0x36 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4] 2> 2363 T583 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 2363 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a40003 type:create cxid:0x73 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 2364 T583 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 2365 T583 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 2365 T583 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:46561/collection1/
[junit4:junit4] 2> 2365 T583 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 2365 T583 oasc.SyncStrategy.syncToMe http://127.0.0.1:46561/collection1/ has no replicas
[junit4:junit4] 2> 2365 T583 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:46561/collection1/
[junit4:junit4] 2> 2365 T583 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 2368 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a40003 type:create cxid:0x7d zxid:0x3b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 3482 T581 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 3488 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 3522 T583 oasc.ZkController.register We are http://127.0.0.1:46561/collection1/ and leader is http://127.0.0.1:46561/collection1/
[junit4:junit4] 2> 3522 T583 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:46561
[junit4:junit4] 2> 3523 T583 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 3523 T583 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 3523 T583 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 3524 T583 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 3526 T558 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 3526 T558 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 3526 T558 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 3529 T558 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 3530 T558 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54542/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@81702b
[junit4:junit4] 2> 3531 T558 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 3531 T586 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54542. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 3531 T586 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54542, initiating session
[junit4:junit4] 2> 3531 T560 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:42446
[junit4:junit4] 2> 3532 T560 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:42446
[junit4:junit4] 2> 3532 T562 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d2d11d4a40004 with negotiated timeout 10000 for client /127.0.0.1:42446
[junit4:junit4] 2> 3532 T586 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54542, sessionid = 0x13d2d11d4a40004, negotiated timeout = 10000
[junit4:junit4] 2> 3533 T587 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@81702b name:ZooKeeperConnection Watcher:127.0.0.1:54542/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 3533 T558 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 3535 T558 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 3537 T558 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 3605 T558 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 3607 T558 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:46361
[junit4:junit4] 2> 3607 T558 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 3608 T558 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 3608 T558 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362260779631
[junit4:junit4] 2> 3608 T558 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362260779631/solr.xml
[junit4:junit4] 2> 3609 T558 oasc.CoreContainer.<init> New CoreContainer 19300167
[junit4:junit4] 2> 3609 T558 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362260779631/'
[junit4:junit4] 2> 3610 T558 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362260779631/'
[junit4:junit4] 2> 3641 T558 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 3642 T558 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 3642 T558 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 3642 T558 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 3643 T558 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 3643 T558 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 3643 T558 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 3644 T558 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 3644 T558 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 3644 T558 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 3654 T558 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 3670 T558 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:54542/solr
[junit4:junit4] 2> 3670 T558 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 3671 T558 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54542 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@db4ce1
[junit4:junit4] 2> 3672 T558 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 3672 T597 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54542. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 3672 T597 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54542, initiating session
[junit4:junit4] 2> 3673 T560 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:42447
[junit4:junit4] 2> 3673 T560 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:42447
[junit4:junit4] 2> 3674 T562 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d2d11d4a40005 with negotiated timeout 20000 for client /127.0.0.1:42447
[junit4:junit4] 2> 3674 T597 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54542, sessionid = 0x13d2d11d4a40005, negotiated timeout = 20000
[junit4:junit4] 2> 3674 T598 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@db4ce1 name:ZooKeeperConnection Watcher:127.0.0.1:54542 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 3674 T558 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 3676 T563 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d2d11d4a40005
[junit4:junit4] 2> 3676 T598 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 3676 T560 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:42447 which had sessionid 0x13d2d11d4a40005
[junit4:junit4] 2> 3676 T558 oaz.ZooKeeper.close Session: 0x13d2d11d4a40005 closed
[junit4:junit4] 2> 3677 T558 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 3688 T558 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54542/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@1813de4
[junit4:junit4] 2> 3689 T558 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 3689 T599 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54542. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 3690 T599 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54542, initiating session
[junit4:junit4] 2> 3690 T560 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:42448
[junit4:junit4] 2> 3690 T560 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:42448
[junit4:junit4] 2> 3691 T562 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d2d11d4a40006 with negotiated timeout 20000 for client /127.0.0.1:42448
[junit4:junit4] 2> 3691 T599 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54542, sessionid = 0x13d2d11d4a40006, negotiated timeout = 20000
[junit4:junit4] 2> 3691 T600 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1813de4 name:ZooKeeperConnection Watcher:127.0.0.1:54542/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 3692 T558 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 3693 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a40006 type:create cxid:0x1 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 3693 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a40006 type:create cxid:0x2 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 3694 T558 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 4696 T558 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46361_
[junit4:junit4] 2> 4697 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a40006 type:delete cxid:0x9 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:46361_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:46361_
[junit4:junit4] 2> 4698 T558 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:46361_
[junit4:junit4] 2> 4700 T580 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 4700 T587 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 4700 T600 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 4701 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 4705 T601 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362260779631/collection1
[junit4:junit4] 2> 4705 T601 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 4706 T601 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 4706 T601 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 4707 T601 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362260779631/collection1/'
[junit4:junit4] 2> 4707 T601 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362260779631/collection1/lib/README' to classloader
[junit4:junit4] 2> 4708 T601 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362260779631/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 4744 T601 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 4798 T601 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 4799 T601 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 4803 T601 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 4992 T581 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 4993 T581 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "base_url":"http://127.0.0.1:46561",
[junit4:junit4] 2> "node_name":"127.0.0.1:46561_",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1"}
[junit4:junit4] 2> 5000 T587 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 5000 T600 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 5000 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 5459 T601 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 5468 T601 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 5471 T601 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 5480 T601 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 5486 T601 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 5489 T601 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 5490 T601 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 5490 T601 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 5490 T601 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 5491 T601 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 5491 T601 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 5491 T601 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362260779631/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty1/
[junit4:junit4] 2> 5492 T601 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@e089a3
[junit4:junit4] 2> 5492 T601 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 5493 T601 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty1 forceNew:false
[junit4:junit4] 2> 5493 T601 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty1
[junit4:junit4] 2> 5493 T601 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty1/index/
[junit4:junit4] 2> 5493 T601 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 5494 T601 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty1/index forceNew:false
[junit4:junit4] 2> 5495 T601 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@16fb8ab lockFactory=org.apache.lucene.store.NativeFSLockFactory@119e26e),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 5495 T601 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 5495 T601 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty1/index
[junit4:junit4] 2> 5496 T601 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 5497 T601 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 5497 T601 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 5497 T601 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 5498 T601 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 5498 T601 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 5498 T601 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 5499 T601 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 5499 T601 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 5501 T601 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 5504 T601 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty1
[junit4:junit4] 2> 5504 T601 oass.SolrIndexSearcher.<init> Opening Searcher@1aa1b72 main
[junit4:junit4] 2> 5504 T601 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty1/tlog
[junit4:junit4] 2> 5505 T601 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 5505 T601 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 5511 T602 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1aa1b72 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 5513 T601 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 5513 T601 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 6504 T581 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 6505 T581 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "base_url":"http://127.0.0.1:46361",
[junit4:junit4] 2> "node_name":"127.0.0.1:46361_",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null}
[junit4:junit4] 2> 6505 T581 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4] 2> 6505 T581 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 6509 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 6509 T587 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 6509 T600 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 6514 T601 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 6514 T601 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:46361 collection:collection1 shard:shard1
[junit4:junit4] 2> 6515 T601 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 6519 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a40006 type:delete cxid:0x46 zxid:0x5b txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4] 2> 6520 T601 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 6520 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a40006 type:create cxid:0x47 zxid:0x5c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 6521 T601 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 6521 T601 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 6522 T601 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:46361/collection1/
[junit4:junit4] 2> 6522 T601 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 6522 T601 oasc.SyncStrategy.syncToMe http://127.0.0.1:46361/collection1/ has no replicas
[junit4:junit4] 2> 6522 T601 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:46361/collection1/
[junit4:junit4] 2> 6522 T601 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 6525 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a40006 type:create cxid:0x51 zxid:0x60 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 8013 T581 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 8020 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 8020 T600 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 8020 T587 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 8029 T601 oasc.ZkController.register We are http://127.0.0.1:46361/collection1/ and leader is http://127.0.0.1:46361/collection1/
[junit4:junit4] 2> 8029 T601 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:46361
[junit4:junit4] 2> 8029 T601 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 8029 T601 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 8030 T601 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 8031 T601 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 8032 T558 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 8032 T558 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 8033 T558 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 8104 T558 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 8106 T558 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50003
[junit4:junit4] 2> 8106 T558 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 8107 T558 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 8107 T558 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362260784130
[junit4:junit4] 2> 8107 T558 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362260784130/solr.xml
[junit4:junit4] 2> 8108 T558 oasc.CoreContainer.<init> New CoreContainer 6970208
[junit4:junit4] 2> 8108 T558 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362260784130/'
[junit4:junit4] 2> 8108 T558 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362260784130/'
[junit4:junit4] 2> 8137 T558 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 8137 T558 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 8137 T558 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 8138 T558 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 8138 T558 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 8139 T558 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 8139 T558 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 8139 T558 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 8139 T558 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 8140 T558 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 8157 T558 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 8178 T558 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:54542/solr
[junit4:junit4] 2> 8179 T558 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 8179 T558 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54542 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@125e59e
[junit4:junit4] 2> 8180 T558 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 8180 T613 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54542. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 8181 T613 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54542, initiating session
[junit4:junit4] 2> 8181 T560 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:42452
[junit4:junit4] 2> 8181 T560 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:42452
[junit4:junit4] 2> 8182 T562 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d2d11d4a40007 with negotiated timeout 20000 for client /127.0.0.1:42452
[junit4:junit4] 2> 8182 T613 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54542, sessionid = 0x13d2d11d4a40007, negotiated timeout = 20000
[junit4:junit4] 2> 8182 T614 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@125e59e name:ZooKeeperConnection Watcher:127.0.0.1:54542 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 8183 T558 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 8183 T563 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d2d11d4a40007
[junit4:junit4] 2> 8184 T614 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 8184 T558 oaz.ZooKeeper.close Session: 0x13d2d11d4a40007 closed
[junit4:junit4] 2> 8184 T560 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:42452 which had sessionid 0x13d2d11d4a40007
[junit4:junit4] 2> 8185 T558 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 8187 T558 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54542/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@dcc013
[junit4:junit4] 2> 8188 T558 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 8188 T615 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54542. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 8189 T615 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54542, initiating session
[junit4:junit4] 2> 8189 T560 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:42453
[junit4:junit4] 2> 8189 T560 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:42453
[junit4:junit4] 2> 8190 T562 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d2d11d4a40008 with negotiated timeout 20000 for client /127.0.0.1:42453
[junit4:junit4] 2> 8190 T615 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54542, sessionid = 0x13d2d11d4a40008, negotiated timeout = 20000
[junit4:junit4] 2> 8190 T616 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@dcc013 name:ZooKeeperConnection Watcher:127.0.0.1:54542/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 8190 T558 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 8191 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a40008 type:create cxid:0x1 zxid:0x6d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 8192 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a40008 type:create cxid:0x2 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 8193 T558 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 9195 T558 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50003_
[junit4:junit4] 2> 9196 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a40008 type:delete cxid:0x9 zxid:0x6f txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:50003_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:50003_
[junit4:junit4] 2> 9197 T558 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50003_
[junit4:junit4] 2> 9200 T580 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9200 T587 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9200 T616 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9200 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9200 T587 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9200 T600 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9201 T600 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9205 T617 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362260784130/collection1
[junit4:junit4] 2> 9205 T617 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 9206 T617 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 9206 T617 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 9207 T617 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362260784130/collection1/'
[junit4:junit4] 2> 9207 T617 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362260784130/collection1/lib/README' to classloader
[junit4:junit4] 2> 9207 T617 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362260784130/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 9253 T617 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 9303 T617 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 9304 T617 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 9308 T617 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 9524 T581 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 9525 T581 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "base_url":"http://127.0.0.1:46361",
[junit4:junit4] 2> "node_name":"127.0.0.1:46361_",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1"}
[junit4:junit4] 2> 9535 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9535 T587 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9535 T600 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9535 T616 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9975 T617 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 9985 T617 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 9988 T617 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 10001 T617 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 10004 T617 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 10007 T617 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 10008 T617 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 10009 T617 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 10009 T617 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 10009 T617 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 10010 T617 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 10010 T617 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362260784130/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty2/
[junit4:junit4] 2> 10010 T617 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@e089a3
[junit4:junit4] 2> 10010 T617 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 10011 T617 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty2 forceNew:false
[junit4:junit4] 2> 10011 T617 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty2
[junit4:junit4] 2> 10011 T617 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty2/index/
[junit4:junit4] 2> 10012 T617 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 10012 T617 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty2/index forceNew:false
[junit4:junit4] 2> 10013 T617 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@c91ecc lockFactory=org.apache.lucene.store.NativeFSLockFactory@1f47c19),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 10013 T617 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 10014 T617 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty2/index
[junit4:junit4] 2> 10015 T617 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 10015 T617 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 10015 T617 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 10016 T617 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 10016 T617 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 10017 T617 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 10017 T617 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 10017 T617 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 10017 T617 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 10020 T617 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 10025 T617 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty2
[junit4:junit4] 2> 10026 T617 oass.SolrIndexSearcher.<init> Opening Searcher@13c22c6 main
[junit4:junit4] 2> 10026 T617 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty2/tlog
[junit4:junit4] 2> 10026 T617 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 10027 T617 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 10030 T618 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@13c22c6 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 10032 T617 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 10032 T617 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 11039 T581 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 11040 T581 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50003",
[junit4:junit4] 2> "node_name":"127.0.0.1:50003_",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null}
[junit4:junit4] 2> 11040 T581 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 11041 T581 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4] 2> 11045 T600 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 11045 T616 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 11045 T587 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 11045 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 12033 T617 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 12033 T617 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:50003 collection:collection1 shard:shard2
[junit4:junit4] 2> 12034 T617 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 12039 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a40008 type:delete cxid:0x45 zxid:0x7e txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
[junit4:junit4] 2> 12039 T617 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 12040 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a40008 type:create cxid:0x46 zxid:0x7f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 12041 T617 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 12041 T617 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 12041 T617 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:50003/collection1/
[junit4:junit4] 2> 12042 T617 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 12042 T617 oasc.SyncStrategy.syncToMe http://127.0.0.1:50003/collection1/ has no replicas
[junit4:junit4] 2> 12042 T617 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:50003/collection1/
[junit4:junit4] 2> 12042 T617 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 12045 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a40008 type:create cxid:0x4f zxid:0x82 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 12549 T581 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 12555 T616 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 12556 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 12556 T600 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 12556 T587 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 12597 T617 oasc.ZkController.register We are http://127.0.0.1:50003/collection1/ and leader is http://127.0.0.1:50003/collection1/
[junit4:junit4] 2> 12597 T617 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:50003
[junit4:junit4] 2> 12597 T617 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 12598 T617 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 12598 T617 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 12599 T617 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 12600 T558 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 12600 T558 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 12601 T558 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 12671 T558 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 12673 T558 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:44863
[junit4:junit4] 2> 12674 T558 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 12674 T558 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 12674 T558 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362260788698
[junit4:junit4] 2> 12675 T558 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362260788698/solr.xml
[junit4:junit4] 2> 12675 T558 oasc.CoreContainer.<init> New CoreContainer 30062125
[junit4:junit4] 2> 12675 T558 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362260788698/'
[junit4:junit4] 2> 12676 T558 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362260788698/'
[junit4:junit4] 2> 12701 T558 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 12701 T558 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 12702 T558 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 12702 T558 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 12702 T558 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 12703 T558 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 12703 T558 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 12703 T558 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 12704 T558 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 12704 T558 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 12711 T558 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 12732 T558 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:54542/solr
[junit4:junit4] 2> 12733 T558 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 12733 T558 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54542 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@1a15eee
[junit4:junit4] 2> 12734 T629 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54542. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 12734 T558 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 12735 T629 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54542, initiating session
[junit4:junit4] 2> 12735 T560 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:42465
[junit4:junit4] 2> 12735 T560 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:42465
[junit4:junit4] 2> 12736 T562 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d2d11d4a40009 with negotiated timeout 20000 for client /127.0.0.1:42465
[junit4:junit4] 2> 12736 T629 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54542, sessionid = 0x13d2d11d4a40009, negotiated timeout = 20000
[junit4:junit4] 2> 12736 T630 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a15eee name:ZooKeeperConnection Watcher:127.0.0.1:54542 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 12736 T558 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 12738 T563 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d2d11d4a40009
[junit4:junit4] 2> 12738 T630 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 12738 T560 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:42465 which had sessionid 0x13d2d11d4a40009
[junit4:junit4] 2> 12738 T558 oaz.ZooKeeper.close Session: 0x13d2d11d4a40009 closed
[junit4:junit4] 2> 12739 T558 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 12742 T558 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54542/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@12579a9
[junit4:junit4] 2> 12743 T558 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 12743 T631 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54542. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 12744 T631 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54542, initiating session
[junit4:junit4] 2> 12744 T560 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:42466
[junit4:junit4] 2> 12744 T560 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:42466
[junit4:junit4] 2> 12745 T562 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d2d11d4a4000a with negotiated timeout 20000 for client /127.0.0.1:42466
[junit4:junit4] 2> 12745 T631 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54542, sessionid = 0x13d2d11d4a4000a, negotiated timeout = 20000
[junit4:junit4] 2> 12745 T632 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@12579a9 name:ZooKeeperConnection Watcher:127.0.0.1:54542/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 12745 T558 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 12746 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a4000a type:create cxid:0x1 zxid:0x8f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 12748 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a4000a type:create cxid:0x2 zxid:0x90 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 12750 T558 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 13751 T558 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44863_
[junit4:junit4] 2> 13752 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a4000a type:delete cxid:0x9 zxid:0x91 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:44863_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:44863_
[junit4:junit4] 2> 13753 T558 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:44863_
[junit4:junit4] 2> 13755 T600 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 13755 T587 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 13755 T616 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 13755 T580 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 13756 T587 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 13756 T616 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 13756 T632 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 13756 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 13760 T600 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 13768 T633 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362260788698/collection1
[junit4:junit4] 2> 13769 T633 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 13770 T633 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 13770 T633 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 13771 T633 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362260788698/collection1/'
[junit4:junit4] 2> 13772 T633 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362260788698/collection1/lib/README' to classloader
[junit4:junit4] 2> 13772 T633 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362260788698/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 13813 T633 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 13883 T633 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 13884 T633 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 13895 T633 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 14063 T581 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 14064 T581 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50003",
[junit4:junit4] 2> "node_name":"127.0.0.1:50003_",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2"}
[junit4:junit4] 2> 14067 T600 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14067 T616 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14067 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14067 T632 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14067 T587 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14577 T633 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 14585 T633 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 14591 T633 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 14602 T633 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 14605 T633 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 14608 T633 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 14609 T633 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 14610 T633 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 14610 T633 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 14611 T633 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 14611 T633 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 14611 T633 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362260788698/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty3/
[junit4:junit4] 2> 14611 T633 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@e089a3
[junit4:junit4] 2> 14612 T633 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 14612 T633 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty3 forceNew:false
[junit4:junit4] 2> 14612 T633 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty3
[junit4:junit4] 2> 14613 T633 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty3/index/
[junit4:junit4] 2> 14613 T633 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 14613 T633 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty3/index forceNew:false
[junit4:junit4] 2> 14615 T633 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1d6e5e1 lockFactory=org.apache.lucene.store.NativeFSLockFactory@44b1da),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 14615 T633 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 14616 T633 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty3/index
[junit4:junit4] 2> 14622 T633 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 14622 T633 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 14623 T633 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 14623 T633 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 14624 T633 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 14624 T633 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 14624 T633 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 14625 T633 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 14625 T633 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 14628 T633 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 14630 T633 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty3
[junit4:junit4] 2> 14631 T633 oass.SolrIndexSearcher.<init> Opening Searcher@8f46ac main
[junit4:junit4] 2> 14631 T633 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty3/tlog
[junit4:junit4] 2> 14632 T633 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 14632 T633 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 14637 T634 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@8f46ac main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 14638 T633 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 14638 T633 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 15582 T581 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 15583 T581 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "base_url":"http://127.0.0.1:44863",
[junit4:junit4] 2> "node_name":"127.0.0.1:44863_",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null}
[junit4:junit4] 2> 15583 T581 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 15583 T581 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 15588 T616 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 15588 T587 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 15588 T632 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 15588 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 15588 T600 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 15639 T633 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 15640 T633 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:44863 collection:collection1 shard:shard1
[junit4:junit4] 2> 15642 T633 oasc.ZkController.register We are http://127.0.0.1:44863/collection1/ and leader is http://127.0.0.1:46361/collection1/
[junit4:junit4] 2> 15642 T633 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:44863
[junit4:junit4] 2> 15642 T633 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 15642 T633 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C40 name=collection1 org.apache.solr.core.SolrCore@1fef186 url=http://127.0.0.1:44863/collection1 node=127.0.0.1:44863_ C40_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:44863, node_name=127.0.0.1:44863_, collection=collection1, core=collection1, state=down}
[junit4:junit4] 2> 15643 T635 C40 P44863 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 15643 T633 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 15643 T635 C40 P44863 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 15644 T635 C40 P44863 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 15644 T635 C40 P44863 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 15644 T558 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 15645 T635 C40 P44863 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 15645 T558 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 15645 T558 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 15650 T558 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 15651 T558 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 15657 T596 oasc.CoreContainer.create Creating SolrCore 'onenodecollectioncore' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362260779631/onenodecollectioncore
[junit4:junit4] 2> 15657 T596 oasc.ZkController.createCollectionZkNode Check for collection zkNode:onenodecollection
[junit4:junit4] 2> 15658 T596 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:onenodecollection
[junit4:junit4] 2> 15658 T596 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 15659 T596 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 15659 T596 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection
[junit4:junit4] 2> 15661 T596 oasc.ZkController.readConfigName Load collection config from:/collections/onenodecollection
[junit4:junit4] 2> 15662 T596 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362260779631/onenodecollectioncore/'
[junit4:junit4] 2> 15707 T596 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 15771 T596 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 15772 T596 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 15776 T596 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 16463 T596 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 16478 T596 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 16481 T596 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 16491 T596 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 16498 T596 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 16501 T596 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 16502 T596 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 16503 T596 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 16503 T596 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 16504 T596 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 16504 T596 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 16504 T596 oasc.SolrCore.<init> [onenodecollectioncore] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362260779631/onenodecollectioncore/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362260776097/onenodecollection/
[junit4:junit4] 2> 16505 T596 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@e089a3
[junit4:junit4] 2> 16506 T596 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 16506 T596 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362260776097/onenodecollection forceNew:false
[junit4:junit4] 2> 16507 T596 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362260776097/onenodecollection
[junit4:junit4] 2> 16507 T596 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362260776097/onenodecollection/index/
[junit4:junit4] 2> 16507 T596 oasc.SolrCore.initIndex WARNING [onenodecollectioncore] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362260776097/onenodecollection/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 16508 T596 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362260776097/onenodecollection/index forceNew:false
[junit4:junit4] 2> 16509 T596 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@fbdcbe lockFactory=org.apache.lucene.store.NativeFSLockFactory@1975943),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 16510 T596 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 16510 T596 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362260776097/onenodecollection/index
[junit4:junit4] 2> 16511 T596 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 16512 T596 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 16512 T596 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 16513 T596 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 16513 T596 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 16513 T596 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 16514 T596 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 16514 T596 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 16514 T596 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 16517 T596 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 16520 T596 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362260776097/onenodecollection
[junit4:junit4] 2> 16520 T596 oass.SolrIndexSearcher.<init> Opening Searcher@1347eeb main
[junit4:junit4] 2> 16521 T596 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362260776097/onenodecollection/tlog
[junit4:junit4] 2> 16521 T596 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 16521 T596 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 16524 T596 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362260776097/onenodecollection
[junit4:junit4] 2> 16525 T637 oasc.SolrCore.registerSearcher [onenodecollectioncore] Registered new searcher Searcher@1347eeb main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 16529 T596 oasc.ZkController.publish publishing core=onenodecollectioncore state=down
[junit4:junit4] 2> 17096 T581 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 17097 T581 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "base_url":"http://127.0.0.1:44863",
[junit4:junit4] 2> "node_name":"127.0.0.1:44863_",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1"}
[junit4:junit4] 2> 17099 T581 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "base_url":"http://127.0.0.1:46361",
[junit4:junit4] 2> "node_name":"127.0.0.1:46361_",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "core":"onenodecollectioncore",
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null}
[junit4:junit4] 2> 17099 T581 oasc.Overseer$ClusterStateUpdater.createCollection Create collection onenodecollection with numShards 1
[junit4:junit4] 2> 17099 T581 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 17102 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17102 T587 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17102 T616 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17102 T600 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17102 T632 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17531 T596 oasc.CoreContainer.registerCore registering core: onenodecollectioncore
[junit4:junit4] 2> 17531 T596 oasc.ZkController.register Register replica - core:onenodecollectioncore address:http://127.0.0.1:46361 collection:onenodecollection shard:shard1
[junit4:junit4] 2> 17532 T596 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection/leader_elect/shard1/election
[junit4:junit4] 2> 17537 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a40006 type:delete cxid:0x9f zxid:0xac txntype:-1 reqpath:n/a Error Path:/solr/collections/onenodecollection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/onenodecollection/leaders
[junit4:junit4] 2> 17537 T596 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 17538 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a40006 type:create cxid:0xa0 zxid:0xad txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 17539 T596 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 17540 T596 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 17540 T596 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:46361/onenodecollectioncore/
[junit4:junit4] 2> 17540 T596 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 17541 T596 oasc.SyncStrategy.syncToMe http://127.0.0.1:46361/onenodecollectioncore/ has no replicas
[junit4:junit4] 2> 17541 T596 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:46361/onenodecollectioncore/
[junit4:junit4] 2> 17541 T596 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection/leaders/shard1
[junit4:junit4] 2> 17544 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a40006 type:create cxid:0xaa zxid:0xb1 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 18607 T581 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 18614 T600 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 18614 T616 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 18614 T587 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 18614 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 18614 T632 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 18647 T596 oasc.ZkController.register We are http://127.0.0.1:46361/onenodecollectioncore/ and leader is http://127.0.0.1:46361/onenodecollectioncore/
[junit4:junit4] 2> 18648 T596 oasc.ZkController.register No LogReplay needed for core=onenodecollectioncore baseURL=http://127.0.0.1:46361
[junit4:junit4] 2> 18648 T596 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 18649 T596 oasc.ZkController.publish publishing core=onenodecollectioncore state=active
[junit4:junit4] 2> 18651 T596 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 18652 T558 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: onenodecollection failOnTimeout:true timeout (sec):230
[junit4:junit4] 2> 18653 T558 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C40_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:44863, node_name=127.0.0.1:44863_, collection=collection1, core=collection1, state=recovering, shard=shard1}
[junit4:junit4] 2> 19649 T635 C40 P44863 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:46361/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 19650 T635 C40 P44863 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:44863 START replicas=[http://127.0.0.1:46361/collection1/] nUpdates=100
[junit4:junit4] 2> 19650 T635 C40 P44863 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 19650 T635 C40 P44863 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 19650 T635 C40 P44863 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 19650 T635 C40 P44863 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 19651 T635 C40 P44863 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 19651 T635 C40 P44863 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:46361/collection1/. core=collection1
[junit4:junit4] 2> 19651 T635 C40 P44863 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 19656 T558 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C41 name=collection1 org.apache.solr.core.SolrCore@139ba89 url=http://127.0.0.1:46361/collection1 node=127.0.0.1:46361_ C41_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:46361, node_name=127.0.0.1:46361_, collection=collection1, core=collection1, state=active, shard=shard1, leader=true}
[junit4:junit4] 2> 19659 T593 C41 P46361 REQ /get {qt=/get&getVersions=100&version=2&wt=javabin&distrib=false} status=0 QTime=0
[junit4:junit4] 2> 19660 T595 C41 P46361 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 19660 T595 C41 P46361 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty1
[junit4:junit4] 2> 19661 T595 C41 P46361 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@16fb8ab lockFactory=org.apache.lucene.store.NativeFSLockFactory@119e26e),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 19662 T595 C41 P46361 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 19662 T595 C41 P46361 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@16fb8ab lockFactory=org.apache.lucene.store.NativeFSLockFactory@119e26e),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@16fb8ab lockFactory=org.apache.lucene.store.NativeFSLockFactory@119e26e),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> 19662 T595 C41 P46361 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4] 2> 19663 T595 C41 P46361 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty1
[junit4:junit4] 2> 19663 T595 C41 P46361 oass.SolrIndexSearcher.<init> Opening Searcher@fdd6fa realtime
[junit4:junit4] 2> 19664 T595 C41 P46361 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 19664 T595 C41 P46361 /update {openSearcher=false&commit_end_point=true&waitSearcher=true&commit=true&version=2&wt=javabin&softCommit=false} {commit=} 0 4
[junit4:junit4] 2> 19664 T635 C40 P44863 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 19665 T635 C40 P44863 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 19668 T593 C41 P46361 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 19668 T593 C41 P46361 REQ /replication {command=indexversion&qt=/replication&version=2&wt=javabin} status=0 QTime=3
[junit4:junit4] 2> 19669 T635 C40 P44863 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 19669 T635 C40 P44863 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 19669 T635 C40 P44863 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 19671 T596 C41 P46361 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty1
[junit4:junit4] 2> 19671 T596 C41 P46361 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty1/index
[junit4:junit4] 2> 19672 T596 C41 P46361 REQ /replication {command=filelist&qt=/replication&generation=2&version=2&wt=javabin} status=0 QTime=1
[junit4:junit4] 2> 19672 T635 C40 P44863 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 19673 T635 C40 P44863 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty3/index.20130303114635766 forceNew:false
[junit4:junit4] 2> 19673 T635 C40 P44863 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty3
[junit4:junit4] 2> 19673 T635 C40 P44863 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@87973a lockFactory=org.apache.lucene.store.NativeFSLockFactory@62832b) fullCopy=false
[junit4:junit4] 2> 19675 T595 C41 P46361 REQ /replication {command=filecontent&qt=/replication&checksum=true&generation=2&wt=filestream&file=segments_2} status=0 QTime=0
[junit4:junit4] 2> 19686 T635 C40 P44863 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty3/index
[junit4:junit4] 2> 19686 T635 C40 P44863 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 19687 T635 C40 P44863 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty3
[junit4:junit4] 2> 19687 T635 C40 P44863 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty3
[junit4:junit4] 2> 19688 T635 C40 P44863 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 19688 T635 C40 P44863 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 19688 T635 C40 P44863 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty3
[junit4:junit4] 2> 19689 T635 C40 P44863 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1d6e5e1 lockFactory=org.apache.lucene.store.NativeFSLockFactory@44b1da),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1d6e5e1 lockFactory=org.apache.lucene.store.NativeFSLockFactory@44b1da),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> 19689 T635 C40 P44863 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4] 2> 19689 T635 C40 P44863 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 19690 T635 C40 P44863 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty3
[junit4:junit4] 2> 19690 T635 C40 P44863 oass.SolrIndexSearcher.<init> Opening Searcher@74fd29 main
[junit4:junit4] 2> 19691 T634 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@74fd29 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 19691 T634 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty3/index
[junit4:junit4] 2> 19691 T635 C40 P44863 oash.SnapPuller.fetchLatestIndex removing temporary index download directory files MockDirWrapper(org.apache.lucene.store.RAMDirectory@87973a lockFactory=org.apache.lucene.store.NativeFSLockFactory@62832b)
[junit4:junit4] 2> 19691 T635 C40 P44863 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty3/index.20130303114635766
[junit4:junit4] 2> 19692 T635 C40 P44863 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty3/index
[junit4:junit4] 2> 19692 T635 C40 P44863 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 19692 T635 C40 P44863 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 19692 T635 C40 P44863 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 19692 T635 C40 P44863 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 19693 T635 C40 P44863 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 20119 T581 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 20120 T581 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "base_url":"http://127.0.0.1:46361",
[junit4:junit4] 2> "node_name":"127.0.0.1:46361_",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "core":"onenodecollectioncore",
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1"}
[junit4:junit4] 2> 20122 T581 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "base_url":"http://127.0.0.1:44863",
[junit4:junit4] 2> "node_name":"127.0.0.1:44863_",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1"}
[junit4:junit4] 2> 20126 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 20126 T600 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 20126 T616 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 20126 T632 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 20126 T587 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 20658 T558 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 20659 T558 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: onenodecollection
[junit4:junit4] 2> 20659 T558 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> C40_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:44863, node_name=127.0.0.1:44863_, collection=collection1, core=collection1, state=active, shard=shard1}
[junit4:junit4] 2> 20684 T625 C40 P44863 /update {distrib.from=http://127.0.0.1:46361/collection1/&version=2&wt=javabin&update.distrib=FROMLEADER} {add=[1 (1428433977230753792)]} 0 1
[junit4:junit4] 2> 20685 T593 C41 P46361 /update {version=2&wt=javabin} {add=[1 (1428433977230753792)]} 0 12
[junit4:junit4] 2> 20693 T596 C41 P46361 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 20695 T596 C41 P46361 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@16fb8ab lockFactory=org.apache.lucene.store.NativeFSLockFactory@119e26e),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@16fb8ab lockFactory=org.apache.lucene.store.NativeFSLockFactory@119e26e),segFN=segments_3,generation=3,filenames=[_0_nrm.cfs, segments_3, _0_Lucene41_0.doc, _0_Lucene41_0.tim, _0_Lucene41_0.pos, _0_Lucene41_0.tip, _0.fdx, _0_dv.cfe, _0.fdt, _0_nrm.cfe, _0.fnm, _0_dv.cfs, _0.si]
[junit4:junit4] 2> 20696 T596 C41 P46361 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[_0_nrm.cfs, segments_3, _0_Lucene41_0.doc, _0_Lucene41_0.tim, _0_Lucene41_0.pos, _0_Lucene41_0.tip, _0.fdx, _0_dv.cfe, _0.fdt, _0_nrm.cfe, _0.fnm, _0_dv.cfs, _0.si]
[junit4:junit4] 2> 20696 T596 C41 P46361 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty1
[junit4:junit4] 2> 20697 T596 C41 P46361 oass.SolrIndexSearcher.<init> Opening Searcher@56cd4b main
[junit4:junit4] 2> 20697 T596 C41 P46361 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty1/index
[junit4:junit4] 2> 20698 T596 C41 P46361 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 20698 T602 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@56cd4b main{StandardDirectoryReader(segments_3:3:nrt _0(5.0):C1)}
[junit4:junit4] 2> 20699 T602 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty1/index
[junit4:junit4] 2> 20699 T596 C41 P46361 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:44863/collection1/, StdNode: http://127.0.0.1:50003/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 20701 T626 C40 P44863 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 20707 T626 C40 P44863 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1d6e5e1 lockFactory=org.apache.lucene.store.NativeFSLockFactory@44b1da),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1d6e5e1 lockFactory=org.apache.lucene.store.NativeFSLockFactory@44b1da),segFN=segments_3,generation=3,filenames=[_0_nrm.cfs, segments_3, _0_Lucene41_0.doc, _0_Lucene41_0.tim, _0_Lucene41_0.pos, _0_Lucene41_0.tip, _0.fdx, _0_dv.cfe, _0.fdt, _0_nrm.cfe, _0.fnm, _0_dv.cfs, _0.si]
[junit4:junit4] 2> 20708 T626 C40 P44863 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[_0_nrm.cfs, segments_3, _0_Lucene41_0.doc, _0_Lucene41_0.tim, _0_Lucene41_0.pos, _0_Lucene41_0.tip, _0.fdx, _0_dv.cfe, _0.fdt, _0_nrm.cfe, _0.fnm, _0_dv.cfs, _0.si]
[junit4:junit4] 2> 20709 T626 C40 P44863 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty3
[junit4:junit4] 2> 20710 T626 C40 P44863 oass.SolrIndexSearcher.<init> Opening Searcher@1fa8189 main
[junit4:junit4] 2> ASYNC NEW_CORE C42 name=collection1 org.apache.solr.core.SolrCore@3ef117 url=http://127.0.0.1:50003/collection1 node=127.0.0.1:50003_ C42_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:50003, node_name=127.0.0.1:50003_, collection=collection1, core=collection1, state=active, shard=shard2, leader=true}
[junit4:junit4] 2> 20710 T609 C42 P50003 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 20711 T626 C40 P44863 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 20711 T609 C42 P50003 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty2
[junit4:junit4] 2> 20711 T634 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1fa8189 main{StandardDirectoryReader(segments_3:3:nrt _0(5.0):C1)}
[junit4:junit4] 2> 20712 T634 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty3/index
[junit4:junit4] 2> 20712 T626 C40 P44863 /update {commit_end_point=true&expungeDeletes=false&waitSearcher=true&commit=true&version=2&wt=javabin&softCommit=false} {commit=} 0 11
[junit4:junit4] 2> 20712 T609 C42 P50003 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@c91ecc lockFactory=org.apache.lucene.store.NativeFSLockFactory@1f47c19),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 20713 T609 C42 P50003 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 20714 T609 C42 P50003 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@c91ecc lockFactory=org.apache.lucene.store.NativeFSLockFactory@1f47c19),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@c91ecc lockFactory=org.apache.lucene.store.NativeFSLockFactory@1f47c19),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> 20714 T609 C42 P50003 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4] 2> 20715 T609 C42 P50003 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty2
[junit4:junit4] 2> 20715 T609 C42 P50003 oass.SolrIndexSearcher.<init> Opening Searcher@6b13cd main
[junit4:junit4] 2> 20715 T609 C42 P50003 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 20716 T618 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6b13cd main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 20716 T618 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty2/index
[junit4:junit4] 2> 20717 T609 C42 P50003 /update {commit_end_point=true&expungeDeletes=false&waitSearcher=true&commit=true&version=2&wt=javabin&softCommit=false} {commit=} 0 7
[junit4:junit4] 2> 20717 T596 C41 P46361 /update {waitSearcher=true&commit=true&version=2&wt=javabin&softCommit=false} {commit=} 0 24
[junit4:junit4] 2> 20724 T628 C40 P44863 REQ /select {fsv=true&fl=id,score&version=2&wt=javabin&isShard=true&NOW=1362260796814&distrib=false&rows=10&start=0&shard.url=127.0.0.1:46361/collection1/|127.0.0.1:44863/collection1/&q=*:*} hits=1 status=0 QTime=1
[junit4:junit4] 2> 20724 T612 C42 P50003 REQ /select {fsv=true&fl=id,score&version=2&wt=javabin&isShard=true&NOW=1362260796814&distrib=false&rows=10&start=0&shard.url=127.0.0.1:50003/collection1/&q=*:*} hits=0 status=0 QTime=1
[junit4:junit4] 2> 20735 T626 C40 P44863 REQ /select {version=2&wt=javabin&isShard=true&NOW=1362260796814&distrib=false&shard.url=127.0.0.1:46361/collection1/|127.0.0.1:44863/collection1/&q=*:*&ids=1} status=0 QTime=1
[junit4:junit4] 2> 20735 T595 C41 P46361 REQ /select {version=2&wt=javabin&q=*:*} hits=1 status=0 QTime=15
[junit4:junit4] 2> 20737 T558 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C43 name=onenodecollectioncore org.apache.solr.core.SolrCore@70e222 url=http://127.0.0.1:46361/onenodecollectioncore node=127.0.0.1:46361_ C43_STATE=coll:onenodecollection core:onenodecollectioncore props:{base_url=http://127.0.0.1:46361, node_name=127.0.0.1:46361_, collection=onenodecollection, core=onenodecollectioncore, roles=none, state=active, shard=shard1, leader=true}
[junit4:junit4] 2> 20745 T594 C43 P46361 REQ /select {version=2&wt=javabin&q=*:*} hits=0 status=0 QTime=0
[junit4:junit4] 2> 20746 T558 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54542 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@7e41fd
[junit4:junit4] 2> 20747 T558 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 20747 T645 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54542. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 20748 T645 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54542, initiating session
[junit4:junit4] 2> 20748 T560 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:42482
[junit4:junit4] 2> 20748 T560 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:42482
[junit4:junit4] 2> 20749 T562 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d2d11d4a4000b with negotiated timeout 10000 for client /127.0.0.1:42482
[junit4:junit4] 2> 20749 T645 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54542, sessionid = 0x13d2d11d4a4000b, negotiated timeout = 10000
[junit4:junit4] 2> 20749 T646 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7e41fd name:ZooKeeperConnection Watcher:127.0.0.1:54542 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 20750 T558 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 20792 T563 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d2d11d4a4000b
[junit4:junit4] 2> 20792 T646 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 20792 T560 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:42482 which had sessionid 0x13d2d11d4a4000b
[junit4:junit4] 2> 20792 T558 oaz.ZooKeeper.close Session: 0x13d2d11d4a4000b closed
[junit4:junit4] 2> 20793 T563 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d2d11d4a40004
[junit4:junit4] 2> 20793 T587 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 20793 T558 oaz.ZooKeeper.close Session: 0x13d2d11d4a40004 closed
[junit4:junit4] 2> 20793 T560 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:42446 which had sessionid 0x13d2d11d4a40004
[junit4:junit4] 2> 20797 T558 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 46561
[junit4:junit4] 2> 20797 T558 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=11847213
[junit4:junit4] 2> 21630 T581 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 21631 T581 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "core_node_name":"127.0.0.1:46561__collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:46561",
[junit4:junit4] 2> "node_name":"127.0.0.1:46561_",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "shard":"shard1"}
[junit4:junit4] 2> 21634 T632 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 21634 T616 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 21634 T600 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 21634 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 22801 T558 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 22802 T558 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 22804 T558 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@4f18c0
[junit4:junit4] 2> 22805 T558 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty3/index
[junit4:junit4] 2> 22806 T558 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty3
[junit4:junit4] 2> 22812 T558 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 22812 T558 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 22813 T558 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 22813 T558 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 22814 T558 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 22814 T558 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/control/data/index
[junit4:junit4] 2> 22814 T558 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/control/data/index
[junit4:junit4] 2> 22815 T558 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/control/data/index
[junit4:junit4] 2> 22815 T558 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/control/data
[junit4:junit4] 2> 22816 T558 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/control/data
[junit4:junit4] 2> 22816 T581 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89277122222489603-127.0.0.1:46561_-n_0000000000) am no longer a leader.
[junit4:junit4] 2> 22816 T563 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d2d11d4a40003
[junit4:junit4] 2> 22817 T616 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 22818 T560 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:42442 which had sessionid 0x13d2d11d4a40003
[junit4:junit4] 2> 22817 T580 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 22818 T632 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 22818 T558 oaz.ZooKeeper.close Session: 0x13d2d11d4a40003 closed
[junit4:junit4] 2> 22818 T600 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 22818 T580 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 22818 T632 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 22819 T600 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 22819 T580 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 22819 T616 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 22819 T580 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 22821 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a40006 type:delete cxid:0xba zxid:0xcd txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4] 2> 22821 T600 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 22823 T600 oasc.Overseer.start Overseer (id=89277122222489606-127.0.0.1:46361_-n_0000000001) starting
[junit4:junit4] 2> 22823 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a40006 type:create cxid:0xbf zxid:0xcf txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 22824 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a40006 type:create cxid:0xc0 zxid:0xd0 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 22824 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a40006 type:create cxid:0xc1 zxid:0xd1 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 22825 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a40006 type:create cxid:0xc2 zxid:0xd2 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 22826 T648 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 22826 T647 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 22828 T647 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 22828 T647 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "base_url":"http://127.0.0.1:46561",
[junit4:junit4] 2> "node_name":"127.0.0.1:46561_",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1"}
[junit4:junit4] 2> 22832 T632 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 22832 T600 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 22832 T616 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 22840 T558 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
[junit4:junit4] 2> 22891 T558 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 46361
[junit4:junit4] 2> 22892 T558 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=19300167
[junit4:junit4] 2> 24335 T647 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 24336 T647 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "core_node_name":"127.0.0.1:46361__onenodecollectioncore",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:46361",
[junit4:junit4] 2> "node_name":"127.0.0.1:46361_",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "core":"onenodecollectioncore",
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "shard":"shard1"}
[junit4:junit4] 2> 24338 T647 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "core_node_name":"127.0.0.1:46361__collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:46361",
[junit4:junit4] 2> "node_name":"127.0.0.1:46361_",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "shard":"shard1"}
[junit4:junit4] 2> 24341 T600 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 24341 T616 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 24341 T632 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 25894 T558 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 25895 T558 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 25897 T558 oasc.ZkController.publish publishing core=onenodecollectioncore state=down
[junit4:junit4] 2> 25899 T558 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@139ba89
[junit4:junit4] 2> 25904 T558 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty3/index
[junit4:junit4] 2> 25905 T558 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty3
[junit4:junit4] 2> 25907 T558 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=2,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=1,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 25908 T558 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 25908 T558 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 25909 T558 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> C41_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:46361, node_name=127.0.0.1:46361_, collection=collection1, core=collection1, state=down, shard=shard1, leader=true}
[junit4:junit4] 2> 25911 T558 C41 P46361 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty1/index
[junit4:junit4] 2> 25911 T558 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 25911 T558 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty1/index
[junit4:junit4] 2> 25912 T558 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty1/index
[junit4:junit4] 2> 25912 T558 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty1/index
[junit4:junit4] 2> 25912 T558 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty1
[junit4:junit4] 2> 25913 T558 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty1
[junit4:junit4] 2> 25913 T558 oasc.SolrCore.close [onenodecollectioncore] CLOSING SolrCore org.apache.solr.core.SolrCore@70e222
[junit4:junit4] 2> 25919 T558 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 25919 T558 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 25919 T558 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 25920 T558 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 25920 T558 oasc.SolrCore.closeSearcher [onenodecollectioncore] Closing main searcher on request.
[junit4:junit4] 2> 25921 T558 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362260776097/onenodecollection/index
[junit4:junit4] 2> 25921 T558 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362260776097/onenodecollection/index
[junit4:junit4] 2> 25921 T558 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362260776097/onenodecollection/index
[junit4:junit4] 2> 25922 T558 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362260776097/onenodecollection
[junit4:junit4] 2> 25922 T558 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362260776097/onenodecollection
[junit4:junit4] 2> 25923 T647 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89277122222489606-127.0.0.1:46361_-n_0000000001) am no longer a leader.
[junit4:junit4] 2> 25924 T563 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d2d11d4a40006
[junit4:junit4] 2> 25925 T560 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:42448 which had sessionid 0x13d2d11d4a40006
[junit4:junit4] 2> 25925 T600 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 25925 T616 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 25925 T558 oaz.ZooKeeper.close Session: 0x13d2d11d4a40006 closed
[junit4:junit4] 2> 25925 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a4000a type:delete cxid:0x53 zxid:0xe3 txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard1 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard1
[junit4:junit4] 2> 25925 T600 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 25926 T600 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 25926 T632 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 25926 T600 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 25927 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a4000a type:create cxid:0x54 zxid:0xe4 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 25927 T616 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 25928 T632 oasc.ShardLeaderElectionContext.shouldIBeLeader Checking if I should try and be the leader.
[junit4:junit4] 2> 25928 T632 oasc.ShardLeaderElectionContext.shouldIBeLeader My last published State was Active, it's okay to be the leader.
[junit4:junit4] 2> 25928 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a40008 type:delete cxid:0x71 zxid:0xe6 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4] 2> 25932 T632 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 25932 T632 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:44863/collection1/
[junit4:junit4] 2> 25932 T616 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 25932 T632 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:44863 START replicas=[http://127.0.0.1:46361/collection1/] nUpdates=100
[junit4:junit4] 2> 25934 T616 oasc.Overseer.start Overseer (id=89277122222489608-127.0.0.1:50003_-n_0000000002) starting
[junit4:junit4] 2> 25934 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a40008 type:create cxid:0x76 zxid:0xe8 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 25935 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a40008 type:create cxid:0x77 zxid:0xe9 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 25935 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a40008 type:create cxid:0x78 zxid:0xea txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 25936 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a40008 type:create cxid:0x79 zxid:0xeb txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 25937 T632 oasu.PeerSync.handleResponse WARNING PeerSync: core=collection1 url=http://127.0.0.1:44863 couldn't connect to http://127.0.0.1:46361/collection1/, counting as success
[junit4:junit4] 2> 25937 T632 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:44863 DONE. sync succeeded
[junit4:junit4] 2> 25937 T632 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 25937 T632 oasc.SyncStrategy.syncToMe http://127.0.0.1:44863/collection1/: try and ask http://127.0.0.1:46361/collection1/ to sync
[junit4:junit4] 2> 25938 T651 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 25938 T632 oasc.SolrException.log SEVERE Sync request error: org.apache.solr.client.solrj.SolrServerException: Server refused connection at: http://127.0.0.1:46361/collection1
[junit4:junit4] 2> 25938 T632 oasc.SyncStrategy.syncToMe http://127.0.0.1:44863/collection1/: Sync failed - asking replica (http://127.0.0.1:46361/collection1/) to recover.
[junit4:junit4] 2> 25938 T650 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 25939 T632 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:44863/collection1/
[junit4:junit4] 2> 25939 T632 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 25939 T654 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 25940 T650 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 25941 T650 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "base_url":"http://127.0.0.1:46361",
[junit4:junit4] 2> "node_name":"127.0.0.1:46361_",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1"}
[junit4:junit4] 2> 25942 T563 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d11d4a4000a type:create cxid:0x5c zxid:0xee txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 25943 T632 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 25943 T654 oasc.SolrException.log SEVERE http://127.0.0.1:44863/collection1/: Could not tell a replica to recover:org.apache.solr.client.solrj.SolrServerException: Server refused connection at: http://127.0.0.1:46361
[junit4:junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:410)
[junit4:junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181)
[junit4:junit4] 2> at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:298)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2> Caused by: org.apache.http.conn.HttpHostConnectException: Connection to http://127.0.0.1:46361 refused
[junit4:junit4] 2> at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:190)
[junit4:junit4] 2> at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
[junit4:junit4] 2> at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
[junit4:junit4] 2> at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4:junit4] 2> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4:junit4] 2> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4:junit4] 2> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4:junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:353)
[junit4:junit4] 2> ... 5 more
[junit4:junit4] 2> Caused by: java.net.ConnectException: Connection refused
[junit4:junit4] 2> at java.net.PlainSocketImpl.socketConnect(Native Method)
[junit4:junit4] 2> at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:345)
[junit4:junit4] 2> at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
[junit4:junit4] 2> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
[junit4:junit4] 2> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
[junit4:junit4] 2> at java.net.Socket.connect(Socket.java:578)
[junit4:junit4] 2> at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
[junit4:junit4] 2> at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
[junit4:junit4] 2> ... 12 more
[junit4:junit4] 2>
[junit4:junit4] 2> 25944 T632 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 25944 T650 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "base_url":"http://127.0.0.1:46361",
[junit4:junit4] 2> "node_name":"127.0.0.1:46361_",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "core":"onenodecollectioncore",
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1"}
[junit4:junit4] 2> 25947 T558 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
[junit4:junit4] 2> 25952 T632 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 25952 T616 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 25999 T558 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 50003
[junit4:junit4] 2> 25999 T558 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=6970208
[junit4:junit4] 2> 27000 T558 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 27001 T558 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 27003 T558 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@3ef117
[junit4:junit4] 2> 27005 T558 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty3/index
[junit4:junit4] 2> 27005 T558 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty3
[junit4:junit4] 2> 27012 T558 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 27012 T558 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 27013 T558 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 27013 T558 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 27014 T558 C42 P50003 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty2/index
[junit4:junit4] 2> 27015 T558 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 27015 T558 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty2/index
[junit4:junit4] 2> 27015 T558 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty2/index
[junit4:junit4] 2> 27016 T558 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty2/index
[junit4:junit4] 2> 27016 T558 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty2
[junit4:junit4] 2> 27017 T558 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362260776097/jetty2
[junit4:junit4] 2> 27018 T650 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89277122222489608-127.0.0.1:50003_-n_0000000002) am no longer a leader.
[junit4:junit4] 2> 27018 T563 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d2d11d4a40008
[junit4:junit4] 2> 27019 T616 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildre
[...truncated too long message...]
-Dtests.locale=vi -Dtests.timezone=Europe/Uzhgorod -Dtests.file.encoding=UTF-8
[junit4:junit4] FAILURE 135s J1 | UnloadDistributedZkTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: Still found shard
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([F7EF1912CDA91723:7609970ABAF6771F]:0)
[junit4:junit4] > at org.junit.Assert.fail(Assert.java:93)
[junit4:junit4] > at org.apache.solr.cloud.UnloadDistributedZkTest.testUnloadShardAndCollection(UnloadDistributedZkTest.java:124)
[junit4:junit4] > at org.apache.solr.cloud.UnloadDistributedZkTest.doTest(UnloadDistributedZkTest.java:77)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
[junit4:junit4] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4:junit4] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[junit4:junit4] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4:junit4] > at java.lang.reflect.Method.invoke(Method.java:487)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4] > at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4:junit4] > at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4:junit4] > at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4] > at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
[junit4:junit4] > at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4] > at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4] > at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4] > at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4] > at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4] > at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4] > at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2> 134789 T3576 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 134797 T3575 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4:junit4] 2> NOTE: test params are: codec=Lucene40, sim=DefaultSimilarity, locale=vi, timezone=Europe/Uzhgorod
[junit4:junit4] 2> NOTE: Linux 3.2.0-38-generic i386/Oracle Corporation 1.8.0-ea (32-bit)/cpus=8,threads=2,free=138434184,total=273047552
[junit4:junit4] 2> NOTE: All tests run in this JVM: [SoftAutoCommitTest, ZkNodePropsTest, TestDistributedSearch, ClusterStateUpdateTest, CurrencyFieldOpenExchangeTest, TestLuceneMatchVersion, MultiTermTest, FieldMutatingUpdateProcessorTest, StatelessScriptUpdateProcessorFactoryTest, NotRequiredUniqueKeyTest, DirectSolrSpellCheckerTest, NoCacheHeaderTest, SolrIndexSplitterTest, TestLazyCores, AlternateDirectoryTest, TestPerFieldSimilarity, BinaryUpdateRequestHandlerTest, LukeRequestHandlerTest, BasicFunctionalityTest, RequiredFieldsTest, SolrInfoMBeanTest, TestPropInjectDefaults, FullSolrCloudDistribCmdsTest, TestWriterPerf, DirectUpdateHandlerTest, TestMultiCoreConfBootstrap, CopyFieldTest, FieldAnalysisRequestHandlerTest, DistributedQueryElevationComponentTest, OverseerCollectionProcessorTest, IndexSchemaTest, ZkSolrClientTest, TestStressVersions, RequestHandlersTest, TestSolrDeletionPolicy1, TestRealTimeGet, TestRemoteStreaming, TestLMJelinekMercerSimilarityFactory, TestReversedWildcardFilterFactory, TestSuggestSpellingConverter, TestQueryTypes, ExternalFileFieldSortTest, SolrCoreTest, TermVectorComponentTest, ChaosMonkeyNothingIsSafeTest, IndexSchemaRuntimeFieldTest, TimeZoneUtilsTest, HighlighterTest, URLClassifyProcessorTest, AnalysisAfterCoreReloadTest, TestXIncludeConfig, CollectionsAPIDistributedZkTest, RecoveryZkTest, TestRandomDVFaceting, CircularListTest, DateFieldTest, PeerSyncTest, TestQueryUtils, BasicZkTest, BadIndexSchemaTest, TestCSVResponseWriter, TestSolrDeletionPolicy2, OverseerTest, SpellCheckComponentTest, TestReplicationHandler, LeaderElectionTest, DistributedSpellCheckComponentTest, TestDistributedGrouping, TestFaceting, TestHashPartitioner, StatsComponentTest, TestRecovery, TestRangeQuery, TestCoreContainer, TestGroupingSearch, DirectSolrConnectionTest, TestCSVLoader, DocumentBuilderTest, TestPseudoReturnFields, StandardRequestHandlerTest, PingRequestHandlerTest, JsonLoaderTest, DocumentAnalysisRequestHandlerTest, TestSolrQueryParser, JSONWriterTest, SearchHandlerTest, TestArbitraryIndexDir, SuggesterTSTTest, UniqFieldsUpdateProcessorFactoryTest, UpdateParamsTest, CSVRequestHandlerTest, SolrCoreCheckLockOnStartupTest, TestUpdate, TestDocSet, TestLFUCache, IndexReaderFactoryTest, OutputWriterTest, TestConfig, TestLMDirichletSimilarityFactory, LegacyHTMLStripCharFilterTest, TestIBSimilarityFactory, TestFastLRUCache, ResourceLoaderTest, TestSystemIdResolver, TestBadConfig, UUIDFieldTest, RAMDirectoryFactoryTest, SystemInfoHandlerTest, PathHierarchyTokenizerFactoryTest, TestCharFilters, ClusterStateTest, ShardRoutingCustomTest, ShardRoutingTest, SliceStateTest, SliceStateUpdateTest, UnloadDistributedZkTest]
[junit4:junit4] Completed on J1 in 135.08s, 1 test, 1 failure <<< FAILURES!
[...truncated 11 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:381: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:361: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:183: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:449: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1207: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:870: There were test failures: 260 suites, 1095 tests, 2 failures, 9 ignored (3 assumptions)
Total time: 47 minutes 47 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Description set: Java: 32bit/jdk1.8.0-ea-b78 -client -XX:+UseConcMarkSweepGC
Email was triggered for: Failure
Sending email for trigger: Failure
Re: [JENKINS] Lucene-Solr-trunk-Linux (32bit/ibm-j9-jdk7) - Build # 4543 - Still Failing!
Posted by Mark Miller <ma...@gmail.com>.
Should be fixed now.
- Mark
On Mar 2, 2013, at 6:10 PM, Policeman Jenkins Server <je...@thetaphi.de> wrote:
> Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/4543/
> Java: 32bit/ibm-j9-jdk7
>
> 1 tests failed.
> FAILED: org.apache.solr.cloud.BasicDistributedZk2Test.testDistribSearch
>
> Error Message:
> expected:<1> but was:<0>
>
> Stack Trace:
> java.lang.AssertionError: expected:<1> but was:<0>
> at __randomizedtesting.SeedInfo.seed([E534325F463786F:8FB5CD3D833C1853]:0)
> at org.junit.Assert.fail(Assert.java:93)
> at org.junit.Assert.failNotEquals(Assert.java:647)
> at org.junit.Assert.assertEquals(Assert.java:128)
> at org.junit.Assert.assertEquals(Assert.java:472)
> at org.junit.Assert.assertEquals(Assert.java:456)
> at org.apache.solr.cloud.BasicDistributedZk2Test.testNodeWithoutCollectionForwarding(BasicDistributedZk2Test.java:209)
> at org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:89)
> at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:88)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
> at java.lang.reflect.Method.invoke(Method.java:613)
> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
> at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
> at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
> at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
> at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
> at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
> at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
> at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
> at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
> at java.lang.Thread.run(Thread.java:780)
>
>
>
>
> Build Log:
> [...truncated 9499 lines...]
> [junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZk2Test
> [junit4:junit4] 2> 0 T2708 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /jz/n
> [junit4:junit4] 2> 4 T2708 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
> [junit4:junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362265676210
> [junit4:junit4] 2> 5 T2708 oasc.ZkTestServer.run STARTING ZK TEST SERVER
> [junit4:junit4] 2> 6 T2709 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
> [junit4:junit4] 2> 8 T2709 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
> [junit4:junit4] 2> 8 T2709 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
> [junit4:junit4] 2> 9 T2709 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
> [junit4:junit4] 2> 9 T2709 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
> [junit4:junit4] 2> 10 T2709 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/zookeeper/server1/data/version-2/snapshot.0
> [junit4:junit4] 2> 106 T2708 oasc.ZkTestServer.run start zk server on port:54711
> [junit4:junit4] 2> 107 T2708 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54711 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@333af0cc
> [junit4:junit4] 2> 108 T2714 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54711. Will not attempt to authenticate using SASL (Access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4] 2> 108 T2708 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 108 T2714 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54711, initiating session
> [junit4:junit4] 2> 108 T2710 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:54920
> [junit4:junit4] 2> 109 T2710 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:54920
> [junit4:junit4] 2> 109 T2712 oazsp.FileTxnLog.append Creating new log file: log.1
> [junit4:junit4] 2> 126 T2712 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d2d5c99b70000 with negotiated timeout 10000 for client /127.0.0.1:54920
> [junit4:junit4] 2> 126 T2714 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54711, sessionid = 0x13d2d5c99b70000, negotiated timeout = 10000
> [junit4:junit4] 2> 127 T2715 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@333af0cc name:ZooKeeperConnection Watcher:127.0.0.1:54711 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 127 T2708 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 128 T2708 oascc.SolrZkClient.makePath makePath: /solr
> [junit4:junit4] 2> 130 T2713 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d2d5c99b70000
> [junit4:junit4] 2> 131 T2715 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4] 2> 131 T2710 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:54920 which had sessionid 0x13d2d5c99b70000
> [junit4:junit4] 2> 131 T2708 oaz.ZooKeeper.close Session: 0x13d2d5c99b70000 closed
> [junit4:junit4] 2> 132 T2708 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54711/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@6aa2ff8a
> [junit4:junit4] 2> 133 T2716 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54711. Will not attempt to authenticate using SASL (Access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4] 2> 133 T2708 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 133 T2710 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:54921
> [junit4:junit4] 2> 133 T2716 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54711, initiating session
> [junit4:junit4] 2> 134 T2710 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:54921
> [junit4:junit4] 2> 134 T2712 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d2d5c99b70001 with negotiated timeout 10000 for client /127.0.0.1:54921
> [junit4:junit4] 2> 135 T2716 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54711, sessionid = 0x13d2d5c99b70001, negotiated timeout = 10000
> [junit4:junit4] 2> 135 T2717 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6aa2ff8a name:ZooKeeperConnection Watcher:127.0.0.1:54711/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 135 T2708 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 136 T2708 oascc.SolrZkClient.makePath makePath: /collections/collection1
> [junit4:junit4] 2> 138 T2708 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
> [junit4:junit4] 2> 140 T2708 oascc.SolrZkClient.makePath makePath: /collections/control_collection
> [junit4:junit4] 2> 141 T2708 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
> [junit4:junit4] 2> 143 T2708 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
> [junit4:junit4] 2> 144 T2708 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
> [junit4:junit4] 2> 146 T2708 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
> [junit4:junit4] 2> 147 T2708 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
> [junit4:junit4] 2> 150 T2708 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
> [junit4:junit4] 2> 150 T2708 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
> [junit4:junit4] 2> 152 T2708 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
> [junit4:junit4] 2> 153 T2708 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
> [junit4:junit4] 2> 155 T2708 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
> [junit4:junit4] 2> 156 T2708 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
> [junit4:junit4] 2> 159 T2708 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
> [junit4:junit4] 2> 159 T2708 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
> [junit4:junit4] 2> 162 T2708 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
> [junit4:junit4] 2> 163 T2708 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
> [junit4:junit4] 2> 165 T2708 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
> [junit4:junit4] 2> 166 T2708 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
> [junit4:junit4] 2> 168 T2708 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
> [junit4:junit4] 2> 169 T2708 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
> [junit4:junit4] 2> 171 T2713 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d2d5c99b70001
> [junit4:junit4] 2> 172 T2717 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4] 2> 172 T2710 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:54921 which had sessionid 0x13d2d5c99b70001
> [junit4:junit4] 2> 172 T2708 oaz.ZooKeeper.close Session: 0x13d2d5c99b70001 closed
> [junit4:junit4] 2> 232 T2708 oejs.Server.doStart jetty-8.1.8.v20121106
> [junit4:junit4] 2> 245 T2708 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:34136
> [junit4:junit4] 2> 246 T2708 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4:junit4] 2> 246 T2708 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4:junit4] 2> 247 T2708 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362265676378
> [junit4:junit4] 2> 247 T2708 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362265676378/solr.xml
> [junit4:junit4] 2> 248 T2708 oasc.CoreContainer.<init> New CoreContainer 1112915197
> [junit4:junit4] 2> 248 T2708 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362265676378/'
> [junit4:junit4] 2> 249 T2708 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362265676378/'
> [junit4:junit4] 2> 260 T2708 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
> [junit4:junit4] 2> 261 T2708 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4:junit4] 2> 261 T2708 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4:junit4] 2> 262 T2708 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4:junit4] 2> 262 T2708 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4:junit4] 2> 263 T2708 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4:junit4] 2> 264 T2708 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4:junit4] 2> 264 T2708 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4:junit4] 2> 265 T2708 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4:junit4] 2> 265 T2708 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
> [junit4:junit4] 2> 272 T2708 oasc.CoreContainer.load Registering Log Listener
> [junit4:junit4] 2> 282 T2708 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:54711/solr
> [junit4:junit4] 2> 282 T2708 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4:junit4] 2> 283 T2708 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54711 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@6954bf70
> [junit4:junit4] 2> 284 T2727 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54711. Will not attempt to authenticate using SASL (Access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4] 2> 284 T2708 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 284 T2727 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54711, initiating session
> [junit4:junit4] 2> 285 T2710 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:54922
> [junit4:junit4] 2> 285 T2710 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:54922
> [junit4:junit4] 2> 286 T2712 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d2d5c99b70002 with negotiated timeout 20000 for client /127.0.0.1:54922
> [junit4:junit4] 2> 286 T2727 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54711, sessionid = 0x13d2d5c99b70002, negotiated timeout = 20000
> [junit4:junit4] 2> 287 T2728 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6954bf70 name:ZooKeeperConnection Watcher:127.0.0.1:54711 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 287 T2708 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 288 T2713 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d2d5c99b70002
> [junit4:junit4] 2> 289 T2728 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4] 2> 289 T2710 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:54922 which had sessionid 0x13d2d5c99b70002
> [junit4:junit4] 2> 289 T2708 oaz.ZooKeeper.close Session: 0x13d2d5c99b70002 closed
> [junit4:junit4] 2> 290 T2708 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
> [junit4:junit4] 2> 294 T2708 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54711/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@cf20b951
> [junit4:junit4] 2> 295 T2729 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54711. Will not attempt to authenticate using SASL (Access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4] 2> 295 T2708 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 295 T2729 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54711, initiating session
> [junit4:junit4] 2> 296 T2710 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:54923
> [junit4:junit4] 2> 296 T2710 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:54923
> [junit4:junit4] 2> 297 T2712 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d2d5c99b70003 with negotiated timeout 20000 for client /127.0.0.1:54923
> [junit4:junit4] 2> 297 T2729 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54711, sessionid = 0x13d2d5c99b70003, negotiated timeout = 20000
> [junit4:junit4] 2> 298 T2730 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@cf20b951 name:ZooKeeperConnection Watcher:127.0.0.1:54711/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 298 T2708 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 299 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 300 T2708 oascc.SolrZkClient.makePath makePath: /live_nodes
> [junit4:junit4] 2> 302 T2708 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34136_jz%2Fn
> [junit4:junit4] 2> 303 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70003 type:delete cxid:0x8 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:34136_jz%2Fn Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:34136_jz%2Fn
> [junit4:junit4] 2> 303 T2708 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:34136_jz%2Fn
> [junit4:junit4] 2> 305 T2708 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
> [junit4:junit4] 2> 308 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70003 type:delete cxid:0x16 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
> [junit4:junit4] 2> 308 T2708 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
> [junit4:junit4] 2> 311 T2708 oasc.Overseer.start Overseer (id=89277443356426243-127.0.0.1:34136_jz%2Fn-n_0000000000) starting
> [junit4:junit4] 2> 312 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70003 type:create cxid:0x1b zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 313 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70003 type:create cxid:0x1c zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 313 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70003 type:create cxid:0x1d zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 314 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70003 type:create cxid:0x1e zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 315 T2732 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
> [junit4:junit4] 2> 315 T2708 oascc.SolrZkClient.makePath makePath: /clusterstate.json
> [junit4:junit4] 2> 317 T2708 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4:junit4] 2> 318 T2731 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
> [junit4:junit4] 2> 320 T2733 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362265676378/collection1
> [junit4:junit4] 2> 320 T2733 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
> [junit4:junit4] 2> 321 T2733 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4:junit4] 2> 321 T2733 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
> [junit4:junit4] 2> 322 T2733 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362265676378/collection1/'
> [junit4:junit4] 2> 322 T2733 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362265676378/collection1/lib/README' to classloader
> [junit4:junit4] 2> 323 T2733 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362265676378/collection1/lib/classes/' to classloader
> [junit4:junit4] 2> 347 T2733 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4:junit4] 2> 378 T2733 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4:junit4] 2> 379 T2733 oass.IndexSchema.readSchema Reading Solr Schema
> [junit4:junit4] 2> 383 T2733 oass.IndexSchema.readSchema Schema name=test
> [junit4:junit4] 2> 691 T2733 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4:junit4] 2> 695 T2733 oass.IndexSchema.readSchema default search field in schema is text
> [junit4:junit4] 2> 698 T2733 oass.IndexSchema.readSchema unique key field: id
> [junit4:junit4] 2> 707 T2733 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 710 T2733 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 713 T2733 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 714 T2733 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 714 T2733 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
> [junit4:junit4] 2> 715 T2733 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 716 T2733 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 716 T2733 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
> [junit4:junit4] 2> 716 T2733 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362265676378/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/control/data/
> [junit4:junit4] 2> 717 T2733 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3de91d1e
> [junit4:junit4] 2> 717 T2733 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
> [junit4:junit4] 2> 718 T2733 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/control/data forceNew:false
> [junit4:junit4] 2> 718 T2733 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/control/data
> [junit4:junit4] 2> 719 T2733 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/control/data/index/
> [junit4:junit4] 2> 719 T2733 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/control/data/index' doesn't exist. Creating new index...
> [junit4:junit4] 2> 719 T2733 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/control/data/index forceNew:false
> [junit4:junit4] 2> 721 T2733 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@beba4872 lockFactory=org.apache.lucene.store.NativeFSLockFactory@ca29a6eb),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4] 2> 721 T2733 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
> [junit4:junit4] 2> 721 T2733 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/control/data/index
> [junit4:junit4] 2> 723 T2733 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4:junit4] 2> 723 T2733 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4:junit4] 2> 723 T2733 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4:junit4] 2> 724 T2733 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4:junit4] 2> 724 T2733 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4:junit4] 2> 725 T2733 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4:junit4] 2> 725 T2733 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4:junit4] 2> 725 T2733 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4:junit4] 2> 726 T2733 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4:junit4] 2> 727 T2733 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4:junit4] 2> 728 T2733 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/control/data
> [junit4:junit4] 2> 729 T2733 oass.SolrIndexSearcher.<init> Opening Searcher@2581835 main
> [junit4:junit4] 2> 729 T2733 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/control/data/tlog
> [junit4:junit4] 2> 730 T2733 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4:junit4] 2> 730 T2733 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4:junit4] 2> 734 T2734 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2581835 main{StandardDirectoryReader(segments_1:1)}
> [junit4:junit4] 2> 735 T2733 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4] 2> 735 T2733 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 736 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70003 type:create cxid:0x50 zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
> [junit4:junit4] 2> 1821 T2731 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 1821 T2731 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:34136/jz/n",
> [junit4:junit4] 2> "state":"down",
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"1",
> [junit4:junit4] 2> "node_name":"127.0.0.1:34136_jz%2Fn",
> [junit4:junit4] 2> "numShards":"1",
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "collection":"control_collection",
> [junit4:junit4] 2> "shard":null,
> [junit4:junit4] 2> "core":"collection1"}
> [junit4:junit4] 2> 1822 T2731 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
> [junit4:junit4] 2> 1822 T2731 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
> [junit4:junit4] 2> 1822 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70003 type:create cxid:0x58 zxid:0x2c txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
> [junit4:junit4] 2> 1827 T2730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4:junit4] 2> 2737 T2733 oasc.CoreContainer.registerCore registering core: collection1
> [junit4:junit4] 2> 2738 T2733 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:34136/jz/n collection:control_collection shard:shard1
> [junit4:junit4] 2> 2739 T2733 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
> [junit4:junit4] 2> 2743 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70003 type:delete cxid:0x72 zxid:0x36 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
> [junit4:junit4] 2> 2744 T2733 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
> [junit4:junit4] 2> 2744 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70003 type:create cxid:0x73 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 2746 T2733 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
> [junit4:junit4] 2> 2746 T2733 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
> [junit4:junit4] 2> 2746 T2733 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:34136/jz/n/collection1/
> [junit4:junit4] 2> 2747 T2733 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
> [junit4:junit4] 2> 2747 T2733 oasc.SyncStrategy.syncToMe http://127.0.0.1:34136/jz/n/collection1/ has no replicas
> [junit4:junit4] 2> 2747 T2733 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:34136/jz/n/collection1/
> [junit4:junit4] 2> 2748 T2733 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
> [junit4:junit4] 2> 2750 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70003 type:create cxid:0x7d zxid:0x3b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 3332 T2731 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 3343 T2730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4:junit4] 2> 3353 T2733 oasc.ZkController.register We are http://127.0.0.1:34136/jz/n/collection1/ and leader is http://127.0.0.1:34136/jz/n/collection1/
> [junit4:junit4] 2> 3353 T2733 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:34136/jz/n
> [junit4:junit4] 2> 3354 T2733 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
> [junit4:junit4] 2> 3354 T2733 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4:junit4] 2> 3355 T2733 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 3357 T2733 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 3359 T2708 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
> [junit4:junit4] 2> 3360 T2708 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4:junit4] 2> 3362 T2708 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> 3368 T2708 oascsi.HttpClientUtil.createClient Creating new http client, config:
> [junit4:junit4] 2> 3369 T2708 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54711/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@30f40218
> [junit4:junit4] 2> 3370 T2736 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54711. Will not attempt to authenticate using SASL (Access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4] 2> 3370 T2708 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 3371 T2710 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:54928
> [junit4:junit4] 2> 3370 T2736 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54711, initiating session
> [junit4:junit4] 2> 3371 T2710 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:54928
> [junit4:junit4] 2> 3372 T2712 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d2d5c99b70004 with negotiated timeout 10000 for client /127.0.0.1:54928
> [junit4:junit4] 2> 3372 T2736 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54711, sessionid = 0x13d2d5c99b70004, negotiated timeout = 10000
> [junit4:junit4] 2> 3373 T2737 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@30f40218 name:ZooKeeperConnection Watcher:127.0.0.1:54711/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 3373 T2708 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 3374 T2708 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4:junit4] 2> 3375 T2708 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
> [junit4:junit4] 2> 3436 T2708 oejs.Server.doStart jetty-8.1.8.v20121106
> [junit4:junit4] 2> 3438 T2708 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:60021
> [junit4:junit4] 2> 3439 T2708 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4:junit4] 2> 3440 T2708 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4:junit4] 2> 3440 T2708 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362265679580
> [junit4:junit4] 2> 3441 T2708 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362265679580/solr.xml
> [junit4:junit4] 2> 3441 T2708 oasc.CoreContainer.<init> New CoreContainer -1556019191
> [junit4:junit4] 2> 3442 T2708 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362265679580/'
> [junit4:junit4] 2> 3442 T2708 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362265679580/'
> [junit4:junit4] 2> 3455 T2708 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
> [junit4:junit4] 2> 3456 T2708 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4:junit4] 2> 3456 T2708 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4:junit4] 2> 3457 T2708 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4:junit4] 2> 3457 T2708 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4:junit4] 2> 3458 T2708 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4:junit4] 2> 3458 T2708 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4:junit4] 2> 3459 T2708 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4:junit4] 2> 3459 T2708 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4:junit4] 2> 3460 T2708 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
> [junit4:junit4] 2> 3467 T2708 oasc.CoreContainer.load Registering Log Listener
> [junit4:junit4] 2> 3478 T2708 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:54711/solr
> [junit4:junit4] 2> 3478 T2708 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4:junit4] 2> 3479 T2708 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54711 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@73cd271d
> [junit4:junit4] 2> 3480 T2747 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54711. Will not attempt to authenticate using SASL (Access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4] 2> 3480 T2708 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 3480 T2747 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54711, initiating session
> [junit4:junit4] 2> 3481 T2710 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:54929
> [junit4:junit4] 2> 3481 T2710 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:54929
> [junit4:junit4] 2> 3482 T2712 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d2d5c99b70005 with negotiated timeout 20000 for client /127.0.0.1:54929
> [junit4:junit4] 2> 3482 T2747 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54711, sessionid = 0x13d2d5c99b70005, negotiated timeout = 20000
> [junit4:junit4] 2> 3482 T2748 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@73cd271d name:ZooKeeperConnection Watcher:127.0.0.1:54711 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 3483 T2708 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 3484 T2713 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d2d5c99b70005
> [junit4:junit4] 2> 3484 T2748 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4] 2> 3484 T2710 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:54929 which had sessionid 0x13d2d5c99b70005
> [junit4:junit4] 2> 3484 T2708 oaz.ZooKeeper.close Session: 0x13d2d5c99b70005 closed
> [junit4:junit4] 2> 3485 T2708 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
> [junit4:junit4] 2> 3489 T2708 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54711/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@996c3c56
> [junit4:junit4] 2> 3491 T2749 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54711. Will not attempt to authenticate using SASL (Access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4] 2> 3490 T2708 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 3491 T2749 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54711, initiating session
> [junit4:junit4] 2> 3491 T2710 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:54930
> [junit4:junit4] 2> 3492 T2710 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:54930
> [junit4:junit4] 2> 3492 T2712 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d2d5c99b70006 with negotiated timeout 20000 for client /127.0.0.1:54930
> [junit4:junit4] 2> 3492 T2749 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54711, sessionid = 0x13d2d5c99b70006, negotiated timeout = 20000
> [junit4:junit4] 2> 3493 T2750 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@996c3c56 name:ZooKeeperConnection Watcher:127.0.0.1:54711/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 3493 T2708 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 3494 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70006 type:create cxid:0x1 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 3495 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70006 type:create cxid:0x2 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 3496 T2708 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4:junit4] 2> 4498 T2708 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60021_jz%2Fn
> [junit4:junit4] 2> 4499 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70006 type:delete cxid:0x9 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:60021_jz%2Fn Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:60021_jz%2Fn
> [junit4:junit4] 2> 4500 T2708 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:60021_jz%2Fn
> [junit4:junit4] 2> 4502 T2737 oascc.ZkStateReader$3.process Updating live nodes... (2)
> [junit4:junit4] 2> 4502 T2750 oascc.ZkStateReader$3.process Updating live nodes... (2)
> [junit4:junit4] 2> 4502 T2730 oascc.ZkStateReader$3.process Updating live nodes... (2)
> [junit4:junit4] 2> 4502 T2730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 4506 T2751 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362265679580/collection1
> [junit4:junit4] 2> 4506 T2751 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
> [junit4:junit4] 2> 4507 T2751 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4:junit4] 2> 4507 T2751 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
> [junit4:junit4] 2> 4508 T2751 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362265679580/collection1/'
> [junit4:junit4] 2> 4508 T2751 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362265679580/collection1/lib/README' to classloader
> [junit4:junit4] 2> 4509 T2751 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362265679580/collection1/lib/classes/' to classloader
> [junit4:junit4] 2> 4532 T2751 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4:junit4] 2> 4565 T2751 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4:junit4] 2> 4566 T2751 oass.IndexSchema.readSchema Reading Solr Schema
> [junit4:junit4] 2> 4570 T2751 oass.IndexSchema.readSchema Schema name=test
> [junit4:junit4] 2> 4849 T2731 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 4850 T2731 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:34136/jz/n",
> [junit4:junit4] 2> "state":"active",
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"1",
> [junit4:junit4] 2> "node_name":"127.0.0.1:34136_jz%2Fn",
> [junit4:junit4] 2> "numShards":"1",
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "collection":"control_collection",
> [junit4:junit4] 2> "shard":"shard1",
> [junit4:junit4] 2> "core":"collection1"}
> [junit4:junit4] 2> 4852 T2730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 4852 T2750 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 4852 T2737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 4888 T2751 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4:junit4] 2> 4893 T2751 oass.IndexSchema.readSchema default search field in schema is text
> [junit4:junit4] 2> 4896 T2751 oass.IndexSchema.readSchema unique key field: id
> [junit4:junit4] 2> 4906 T2751 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 4911 T2751 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 4916 T2751 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 4917 T2751 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 4918 T2751 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
> [junit4:junit4] 2> 4918 T2751 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 4920 T2751 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 4920 T2751 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
> [junit4:junit4] 2> 4921 T2751 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362265679580/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1/
> [junit4:junit4] 2> 4921 T2751 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3de91d1e
> [junit4:junit4] 2> 4922 T2751 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
> [junit4:junit4] 2> 4923 T2751 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1 forceNew:false
> [junit4:junit4] 2> 4923 T2751 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1
> [junit4:junit4] 2> 4924 T2751 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1/index/
> [junit4:junit4] 2> 4927 T2751 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1/index' doesn't exist. Creating new index...
> [junit4:junit4] 2> 4928 T2751 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1/index forceNew:false
> [junit4:junit4] 2> 4931 T2751 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@c85e682f lockFactory=org.apache.lucene.store.NativeFSLockFactory@b9f9f41b),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4] 2> 4932 T2751 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
> [junit4:junit4] 2> 4933 T2751 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1/index
> [junit4:junit4] 2> 4936 T2751 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4:junit4] 2> 4937 T2751 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4:junit4] 2> 4938 T2751 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4:junit4] 2> 4940 T2751 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4:junit4] 2> 4941 T2751 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4:junit4] 2> 4942 T2751 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4:junit4] 2> 4943 T2751 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4:junit4] 2> 4945 T2751 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4:junit4] 2> 4946 T2751 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4:junit4] 2> 4947 T2751 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4:junit4] 2> 4949 T2751 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1
> [junit4:junit4] 2> 4950 T2751 oass.SolrIndexSearcher.<init> Opening Searcher@42c76c02 main
> [junit4:junit4] 2> 4950 T2751 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1/tlog
> [junit4:junit4] 2> 4951 T2751 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4:junit4] 2> 4952 T2751 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4:junit4] 2> 4955 T2752 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@42c76c02 main{StandardDirectoryReader(segments_1:1)}
> [junit4:junit4] 2> 4957 T2751 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4] 2> 4957 T2751 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 6355 T2731 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 6356 T2731 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:60021/jz/n",
> [junit4:junit4] 2> "state":"down",
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"2",
> [junit4:junit4] 2> "node_name":"127.0.0.1:60021_jz%2Fn",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard":null,
> [junit4:junit4] 2> "core":"collection1"}
> [junit4:junit4] 2> 6357 T2731 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
> [junit4:junit4] 2> 6357 T2731 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
> [junit4:junit4] 2> 6361 T2750 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 6361 T2737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 6361 T2730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 6958 T2751 oasc.CoreContainer.registerCore registering core: collection1
> [junit4:junit4] 2> 6958 T2751 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:60021/jz/n collection:collection1 shard:shard1
> [junit4:junit4] 2> 6959 T2751 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
> [junit4:junit4] 2> 6964 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70006 type:delete cxid:0x46 zxid:0x5b txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
> [junit4:junit4] 2> 6964 T2751 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
> [junit4:junit4] 2> 6965 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70006 type:create cxid:0x47 zxid:0x5c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 6966 T2751 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
> [junit4:junit4] 2> 6966 T2751 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
> [junit4:junit4] 2> 6966 T2751 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:60021/jz/n/collection1/
> [junit4:junit4] 2> 6967 T2751 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
> [junit4:junit4] 2> 6967 T2751 oasc.SyncStrategy.syncToMe http://127.0.0.1:60021/jz/n/collection1/ has no replicas
> [junit4:junit4] 2> 6967 T2751 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:60021/jz/n/collection1/
> [junit4:junit4] 2> 6968 T2751 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
> [junit4:junit4] 2> 6970 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70006 type:create cxid:0x51 zxid:0x60 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 7865 T2731 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 7896 T2737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 7896 T2730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 7896 T2750 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 7924 T2751 oasc.ZkController.register We are http://127.0.0.1:60021/jz/n/collection1/ and leader is http://127.0.0.1:60021/jz/n/collection1/
> [junit4:junit4] 2> 7925 T2751 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:60021/jz/n
> [junit4:junit4] 2> 7926 T2751 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
> [junit4:junit4] 2> 7926 T2751 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4:junit4] 2> 7927 T2751 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 7929 T2751 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 7931 T2708 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
> [junit4:junit4] 2> 7932 T2708 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4:junit4] 2> 7933 T2708 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> 8005 T2708 oejs.Server.doStart jetty-8.1.8.v20121106
> [junit4:junit4] 2> 8007 T2708 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49462
> [junit4:junit4] 2> 8008 T2708 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4:junit4] 2> 8008 T2708 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4:junit4] 2> 8009 T2708 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362265684150
> [junit4:junit4] 2> 8009 T2708 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362265684150/solr.xml
> [junit4:junit4] 2> 8010 T2708 oasc.CoreContainer.<init> New CoreContainer 1479167609
> [junit4:junit4] 2> 8010 T2708 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362265684150/'
> [junit4:junit4] 2> 8011 T2708 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362265684150/'
> [junit4:junit4] 2> 8022 T2708 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
> [junit4:junit4] 2> 8023 T2708 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4:junit4] 2> 8023 T2708 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4:junit4] 2> 8024 T2708 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4:junit4] 2> 8024 T2708 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4:junit4] 2> 8025 T2708 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4:junit4] 2> 8026 T2708 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4:junit4] 2> 8026 T2708 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4:junit4] 2> 8027 T2708 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4:junit4] 2> 8027 T2708 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
> [junit4:junit4] 2> 8034 T2708 oasc.CoreContainer.load Registering Log Listener
> [junit4:junit4] 2> 8044 T2708 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:54711/solr
> [junit4:junit4] 2> 8044 T2708 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4:junit4] 2> 8045 T2708 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54711 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@b8a0312d
> [junit4:junit4] 2> 8046 T2763 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54711. Will not attempt to authenticate using SASL (Access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4] 2> 8046 T2708 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 8046 T2763 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54711, initiating session
> [junit4:junit4] 2> 8046 T2710 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:54936
> [junit4:junit4] 2> 8047 T2710 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:54936
> [junit4:junit4] 2> 8048 T2712 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d2d5c99b70007 with negotiated timeout 20000 for client /127.0.0.1:54936
> [junit4:junit4] 2> 8048 T2763 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54711, sessionid = 0x13d2d5c99b70007, negotiated timeout = 20000
> [junit4:junit4] 2> 8048 T2764 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b8a0312d name:ZooKeeperConnection Watcher:127.0.0.1:54711 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 8049 T2708 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 8049 T2713 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d2d5c99b70007
> [junit4:junit4] 2> 8050 T2710 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:54936 which had sessionid 0x13d2d5c99b70007
> [junit4:junit4] 2> 8050 T2764 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4] 2> 8050 T2708 oaz.ZooKeeper.close Session: 0x13d2d5c99b70007 closed
> [junit4:junit4] 2> 8051 T2708 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
> [junit4:junit4] 2> 8055 T2708 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54711/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@3300a9fe
> [junit4:junit4] 2> 8056 T2765 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54711. Will not attempt to authenticate using SASL (Access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4] 2> 8056 T2708 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 8056 T2765 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54711, initiating session
> [junit4:junit4] 2> 8056 T2710 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:54937
> [junit4:junit4] 2> 8057 T2710 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:54937
> [junit4:junit4] 2> 8058 T2712 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d2d5c99b70008 with negotiated timeout 20000 for client /127.0.0.1:54937
> [junit4:junit4] 2> 8058 T2765 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54711, sessionid = 0x13d2d5c99b70008, negotiated timeout = 20000
> [junit4:junit4] 2> 8058 T2766 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3300a9fe name:ZooKeeperConnection Watcher:127.0.0.1:54711/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 8059 T2708 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 8059 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70008 type:create cxid:0x1 zxid:0x6d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 8060 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70008 type:create cxid:0x2 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 8061 T2708 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4:junit4] 2> 9063 T2708 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49462_jz%2Fn
> [junit4:junit4] 2> 9064 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70008 type:delete cxid:0x9 zxid:0x6f txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:49462_jz%2Fn Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:49462_jz%2Fn
> [junit4:junit4] 2> 9065 T2708 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49462_jz%2Fn
> [junit4:junit4] 2> 9067 T2737 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4:junit4] 2> 9068 T2750 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4:junit4] 2> 9068 T2766 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4:junit4] 2> 9068 T2730 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4:junit4] 2> 9068 T2737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 9068 T2750 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 9069 T2730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 9074 T2767 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362265684150/collection1
> [junit4:junit4] 2> 9075 T2767 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
> [junit4:junit4] 2> 9075 T2767 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4:junit4] 2> 9076 T2767 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
> [junit4:junit4] 2> 9077 T2767 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362265684150/collection1/'
> [junit4:junit4] 2> 9078 T2767 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362265684150/collection1/lib/README' to classloader
> [junit4:junit4] 2> 9079 T2767 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362265684150/collection1/lib/classes/' to classloader
> [junit4:junit4] 2> 9107 T2767 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4:junit4] 2> 9139 T2767 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4:junit4] 2> 9140 T2767 oass.IndexSchema.readSchema Reading Solr Schema
> [junit4:junit4] 2> 9145 T2767 oass.IndexSchema.readSchema Schema name=test
> [junit4:junit4] 2> 9403 T2731 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 9404 T2731 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:60021/jz/n",
> [junit4:junit4] 2> "state":"active",
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"2",
> [junit4:junit4] 2> "node_name":"127.0.0.1:60021_jz%2Fn",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard":"shard1",
> [junit4:junit4] 2> "core":"collection1"}
> [junit4:junit4] 2> 9406 T2737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 9406 T2766 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 9406 T2750 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 9406 T2730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 9483 T2767 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4:junit4] 2> 9488 T2767 oass.IndexSchema.readSchema default search field in schema is text
> [junit4:junit4] 2> 9491 T2767 oass.IndexSchema.readSchema unique key field: id
> [junit4:junit4] 2> 9505 T2767 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 9510 T2767 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 9513 T2767 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 9514 T2767 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 9514 T2767 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
> [junit4:junit4] 2> 9514 T2767 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 9515 T2767 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 9516 T2767 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
> [junit4:junit4] 2> 9516 T2767 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362265684150/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty2/
> [junit4:junit4] 2> 9516 T2767 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3de91d1e
> [junit4:junit4] 2> 9517 T2767 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
> [junit4:junit4] 2> 9517 T2767 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty2 forceNew:false
> [junit4:junit4] 2> 9518 T2767 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty2
> [junit4:junit4] 2> 9518 T2767 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty2/index/
> [junit4:junit4] 2> 9519 T2767 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty2/index' doesn't exist. Creating new index...
> [junit4:junit4] 2> 9519 T2767 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty2/index forceNew:false
> [junit4:junit4] 2> 9520 T2767 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@e8168904 lockFactory=org.apache.lucene.store.NativeFSLockFactory@87db2c99),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4] 2> 9521 T2767 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
> [junit4:junit4] 2> 9521 T2767 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty2/index
> [junit4:junit4] 2> 9522 T2767 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4:junit4] 2> 9523 T2767 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4:junit4] 2> 9523 T2767 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4:junit4] 2> 9524 T2767 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4:junit4] 2> 9524 T2767 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4:junit4] 2> 9524 T2767 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4:junit4] 2> 9525 T2767 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4:junit4] 2> 9525 T2767 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4:junit4] 2> 9526 T2767 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4:junit4] 2> 9527 T2767 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4:junit4] 2> 9529 T2767 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty2
> [junit4:junit4] 2> 9529 T2767 oass.SolrIndexSearcher.<init> Opening Searcher@32ec4ed4 main
> [junit4:junit4] 2> 9530 T2767 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty2/tlog
> [junit4:junit4] 2> 9530 T2767 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4:junit4] 2> 9531 T2767 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4:junit4] 2> 9535 T2768 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@32ec4ed4 main{StandardDirectoryReader(segments_1:1)}
> [junit4:junit4] 2> 9537 T2767 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4] 2> 9537 T2767 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 10912 T2731 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 10913 T2731 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:49462/jz/n",
> [junit4:junit4] 2> "state":"down",
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"3",
> [junit4:junit4] 2> "node_name":"127.0.0.1:49462_jz%2Fn",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard":null,
> [junit4:junit4] 2> "core":"collection1"}
> [junit4:junit4] 2> 10913 T2731 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
> [junit4:junit4] 2> 10914 T2731 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
> [junit4:junit4] 2> 10917 T2737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 10917 T2766 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 10917 T2750 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 10918 T2730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 11539 T2767 oasc.CoreContainer.registerCore registering core: collection1
> [junit4:junit4] 2> 11539 T2767 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:49462/jz/n collection:collection1 shard:shard2
> [junit4:junit4] 2> 11540 T2767 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
> [junit4:junit4] 2> 11544 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70008 type:delete cxid:0x45 zxid:0x7e txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
> [junit4:junit4] 2> 11545 T2767 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
> [junit4:junit4] 2> 11545 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70008 type:create cxid:0x46 zxid:0x7f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 11547 T2767 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
> [junit4:junit4] 2> 11548 T2767 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
> [junit4:junit4] 2> 11548 T2767 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:49462/jz/n/collection1/
> [junit4:junit4] 2> 11549 T2767 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
> [junit4:junit4] 2> 11549 T2767 oasc.SyncStrategy.syncToMe http://127.0.0.1:49462/jz/n/collection1/ has no replicas
> [junit4:junit4] 2> 11550 T2767 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:49462/jz/n/collection1/
> [junit4:junit4] 2> 11550 T2767 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
> [junit4:junit4] 2> 11553 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70008 type:create cxid:0x4f zxid:0x82 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 12421 T2731 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 12426 T2766 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 12426 T2737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 12426 T2750 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 12426 T2730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 12456 T2767 oasc.ZkController.register We are http://127.0.0.1:49462/jz/n/collection1/ and leader is http://127.0.0.1:49462/jz/n/collection1/
> [junit4:junit4] 2> 12457 T2767 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49462/jz/n
> [junit4:junit4] 2> 12458 T2767 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
> [junit4:junit4] 2> 12458 T2767 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4:junit4] 2> 12459 T2767 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 12461 T2767 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 12462 T2708 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
> [junit4:junit4] 2> 12464 T2708 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4:junit4] 2> 12465 T2708 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> 12541 T2708 oejs.Server.doStart jetty-8.1.8.v20121106
> [junit4:junit4] 2> 12543 T2708 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:32892
> [junit4:junit4] 2> 12544 T2708 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4:junit4] 2> 12544 T2708 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4:junit4] 2> 12545 T2708 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362265688675
> [junit4:junit4] 2> 12545 T2708 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362265688675/solr.xml
> [junit4:junit4] 2> 12546 T2708 oasc.CoreContainer.<init> New CoreContainer 1878696378
> [junit4:junit4] 2> 12547 T2708 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362265688675/'
> [junit4:junit4] 2> 12547 T2708 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362265688675/'
> [junit4:junit4] 2> 12561 T2708 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
> [junit4:junit4] 2> 12562 T2708 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4:junit4] 2> 12563 T2708 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4:junit4] 2> 12563 T2708 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4:junit4] 2> 12564 T2708 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4:junit4] 2> 12564 T2708 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4:junit4] 2> 12565 T2708 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4:junit4] 2> 12566 T2708 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4:junit4] 2> 12566 T2708 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4:junit4] 2> 12567 T2708 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
> [junit4:junit4] 2> 12576 T2708 oasc.CoreContainer.load Registering Log Listener
> [junit4:junit4] 2> 12592 T2708 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:54711/solr
> [junit4:junit4] 2> 12593 T2708 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4:junit4] 2> 12594 T2708 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54711 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@ebb8185d
> [junit4:junit4] 2> 12595 T2779 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54711. Will not attempt to authenticate using SASL (Access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4] 2> 12595 T2708 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 12595 T2779 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54711, initiating session
> [junit4:junit4] 2> 12595 T2710 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:54942
> [junit4:junit4] 2> 12596 T2710 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:54942
> [junit4:junit4] 2> 12597 T2712 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d2d5c99b70009 with negotiated timeout 20000 for client /127.0.0.1:54942
> [junit4:junit4] 2> 12597 T2779 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54711, sessionid = 0x13d2d5c99b70009, negotiated timeout = 20000
> [junit4:junit4] 2> 12598 T2780 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ebb8185d name:ZooKeeperConnection Watcher:127.0.0.1:54711 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 12598 T2708 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 12599 T2713 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d2d5c99b70009
> [junit4:junit4] 2> 12600 T2710 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:54942 which had sessionid 0x13d2d5c99b70009
> [junit4:junit4] 2> 12600 T2708 oaz.ZooKeeper.close Session: 0x13d2d5c99b70009 closed
> [junit4:junit4] 2> 12600 T2780 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4] 2> 12601 T2708 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
> [junit4:junit4] 2> 12605 T2708 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54711/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@9da9e782
> [junit4:junit4] 2> 12606 T2781 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54711. Will not attempt to authenticate using SASL (Access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4] 2> 12606 T2708 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 12607 T2710 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:54943
> [junit4:junit4] 2> 12607 T2781 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54711, initiating session
> [junit4:junit4] 2> 12608 T2710 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:54943
> [junit4:junit4] 2> 12609 T2712 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d2d5c99b7000a with negotiated timeout 20000 for client /127.0.0.1:54943
> [junit4:junit4] 2> 12609 T2781 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54711, sessionid = 0x13d2d5c99b7000a, negotiated timeout = 20000
> [junit4:junit4] 2> 12609 T2782 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@9da9e782 name:ZooKeeperConnection Watcher:127.0.0.1:54711/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 12609 T2708 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 12610 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b7000a type:create cxid:0x1 zxid:0x8f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 12611 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b7000a type:create cxid:0x2 zxid:0x90 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 12612 T2708 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
> [junit4:junit4] 2> 13614 T2708 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:32892_jz%2Fn
> [junit4:junit4] 2> 13617 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b7000a type:delete cxid:0x9 zxid:0x91 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:32892_jz%2Fn Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:32892_jz%2Fn
> [junit4:junit4] 2> 13618 T2708 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:32892_jz%2Fn
> [junit4:junit4] 2> 13623 T2766 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4:junit4] 2> 13623 T2730 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4:junit4] 2> 13623 T2750 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4:junit4] 2> 13623 T2782 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4:junit4] 2> 13624 T2766 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 13624 T2750 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 13624 T2737 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4:junit4] 2> 13626 T2737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 13626 T2730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 13634 T2783 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362265688675/collection1
> [junit4:junit4] 2> 13635 T2783 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
> [junit4:junit4] 2> 13636 T2783 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4:junit4] 2> 13637 T2783 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
> [junit4:junit4] 2> 13639 T2783 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362265688675/collection1/'
> [junit4:junit4] 2> 13640 T2783 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362265688675/collection1/lib/README' to classloader
> [junit4:junit4] 2> 13641 T2783 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362265688675/collection1/lib/classes/' to classloader
> [junit4:junit4] 2> 13680 T2783 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4:junit4] 2> 13712 T2783 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4:junit4] 2> 13713 T2783 oass.IndexSchema.readSchema Reading Solr Schema
> [junit4:junit4] 2> 13718 T2783 oass.IndexSchema.readSchema Schema name=test
> [junit4:junit4] 2> 13931 T2731 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 13931 T2731 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:49462/jz/n",
> [junit4:junit4] 2> "state":"active",
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"3",
> [junit4:junit4] 2> "node_name":"127.0.0.1:49462_jz%2Fn",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard":"shard2",
> [junit4:junit4] 2> "core":"collection1"}
> [junit4:junit4] 2> 13935 T2730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 13935 T2782 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 13935 T2750 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 13935 T2766 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 13937 T2737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 14074 T2783 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4:junit4] 2> 14079 T2783 oass.IndexSchema.readSchema default search field in schema is text
> [junit4:junit4] 2> 14082 T2783 oass.IndexSchema.readSchema unique key field: id
> [junit4:junit4] 2> 14093 T2783 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 14096 T2783 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 14101 T2783 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 14102 T2783 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 14103 T2783 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
> [junit4:junit4] 2> 14103 T2783 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 14105 T2783 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 14105 T2783 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
> [junit4:junit4] 2> 14106 T2783 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362265688675/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3/
> [junit4:junit4] 2> 14106 T2783 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3de91d1e
> [junit4:junit4] 2> 14107 T2783 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
> [junit4:junit4] 2> 14108 T2783 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3 forceNew:false
> [junit4:junit4] 2> 14109 T2783 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3
> [junit4:junit4] 2> 14109 T2783 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3/index/
> [junit4:junit4] 2> 14110 T2783 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3/index' doesn't exist. Creating new index...
> [junit4:junit4] 2> 14111 T2783 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3/index forceNew:false
> [junit4:junit4] 2> 14113 T2783 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@17091b05 lockFactory=org.apache.lucene.store.NativeFSLockFactory@58c80795),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4] 2> 14113 T2783 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
> [junit4:junit4] 2> 14114 T2783 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3/index
> [junit4:junit4] 2> 14116 T2783 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4:junit4] 2> 14116 T2783 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4:junit4] 2> 14117 T2783 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4:junit4] 2> 14118 T2783 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4:junit4] 2> 14119 T2783 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4:junit4] 2> 14120 T2783 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4:junit4] 2> 14120 T2783 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4:junit4] 2> 14121 T2783 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4:junit4] 2> 14122 T2783 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4:junit4] 2> 14123 T2783 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4:junit4] 2> 14126 T2783 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3
> [junit4:junit4] 2> 14127 T2783 oass.SolrIndexSearcher.<init> Opening Searcher@ae3b5894 main
> [junit4:junit4] 2> 14127 T2783 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3/tlog
> [junit4:junit4] 2> 14128 T2783 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4:junit4] 2> 14129 T2783 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4:junit4] 2> 14135 T2784 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@ae3b5894 main{StandardDirectoryReader(segments_1:1)}
> [junit4:junit4] 2> 14138 T2783 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4] 2> 14138 T2783 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 15440 T2731 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 15441 T2731 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:32892/jz/n",
> [junit4:junit4] 2> "state":"down",
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"4",
> [junit4:junit4] 2> "node_name":"127.0.0.1:32892_jz%2Fn",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard":null,
> [junit4:junit4] 2> "core":"collection1"}
> [junit4:junit4] 2> 15441 T2731 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
> [junit4:junit4] 2> 15441 T2731 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
> [junit4:junit4] 2> 15446 T2737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 15446 T2750 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 15446 T2782 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 15446 T2730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 15446 T2766 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 16139 T2783 oasc.CoreContainer.registerCore registering core: collection1
> [junit4:junit4] 2> 16140 T2783 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:32892/jz/n collection:collection1 shard:shard1
> [junit4:junit4] 2> 16143 T2783 oasc.ZkController.register We are http://127.0.0.1:32892/jz/n/collection1/ and leader is http://127.0.0.1:60021/jz/n/collection1/
> [junit4:junit4] 2> 16143 T2783 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:32892/jz/n
> [junit4:junit4] 2> 16144 T2783 oasc.ZkController.checkRecovery Core needs to recover:collection1
> [junit4:junit4] 2> 16144 T2783 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
> [junit4:junit4] 2> ASYNC NEW_CORE C176 name=collection1 org.apache.solr.core.SolrCore@18e6b08e url=http://127.0.0.1:32892/jz/n/collection1 node=127.0.0.1:32892_jz%2Fn C176_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:32892/jz/n, state=down, node_name=127.0.0.1:32892_jz%2Fn, collection=collection1, core=collection1}
> [junit4:junit4] 2> 16145 T2785 C176 P32892 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
> [junit4:junit4] 2> 16145 T2785 C176 P32892 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
> [junit4:junit4] 2> 16145 T2783 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 16145 T2785 C176 P32892 oasc.ZkController.publish publishing core=collection1 state=recovering
> [junit4:junit4] 2> 16146 T2785 C176 P32892 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 16146 T2708 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
> [junit4:junit4] 2> 16147 T2708 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4:junit4] 2> 16148 T2708 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> 16148 T2785 C176 P32892 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> 16154 T2708 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 16155 T2708 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> 16163 T2746 oasc.CoreContainer.create Creating SolrCore 'onenodecollectioncore' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362265679580/onenodecollectioncore
> [junit4:junit4] 2> 16164 T2746 oasc.ZkController.createCollectionZkNode Check for collection zkNode:onenodecollection
> [junit4:junit4] 2> 16165 T2746 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:onenodecollection
> [junit4:junit4] 2> 16166 T2746 oasc.ZkController.getConfName Looking for collection configName
> [junit4:junit4] 2> 16167 T2746 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
> [junit4:junit4] 2> 16168 T2746 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection
> [junit4:junit4] 2> 16170 T2746 oasc.ZkController.readConfigName Load collection config from:/collections/onenodecollection
> [junit4:junit4] 2> 16172 T2746 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362265679580/onenodecollectioncore/'
> [junit4:junit4] 2> 16208 T2746 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4:junit4] 2> 16254 T2746 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4:junit4] 2> 16255 T2746 oass.IndexSchema.readSchema Reading Solr Schema
> [junit4:junit4] 2> 16262 T2746 oass.IndexSchema.readSchema Schema name=test
> [junit4:junit4] 2> 16656 T2746 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4:junit4] 2> 16662 T2746 oass.IndexSchema.readSchema default search field in schema is text
> [junit4:junit4] 2> 16666 T2746 oass.IndexSchema.readSchema unique key field: id
> [junit4:junit4] 2> 16677 T2746 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 16682 T2746 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4] 2> 16686 T2746 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 16688 T2746 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 16688 T2746 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
> [junit4:junit4] 2> 16689 T2746 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4] 2> 16690 T2746 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
> [junit4:junit4] 2> 16691 T2746 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
> [junit4:junit4] 2> 16692 T2746 oasc.SolrCore.<init> [onenodecollectioncore] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362265679580/onenodecollectioncore/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362265676210/onenodecollection/
> [junit4:junit4] 2> 16693 T2746 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3de91d1e
> [junit4:junit4] 2> 16694 T2746 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
> [junit4:junit4] 2> 16695 T2746 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362265676210/onenodecollection forceNew:false
> [junit4:junit4] 2> 16696 T2746 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362265676210/onenodecollection
> [junit4:junit4] 2> 16696 T2746 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362265676210/onenodecollection/index/
> [junit4:junit4] 2> 16697 T2746 oasc.SolrCore.initIndex WARNING [onenodecollectioncore] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362265676210/onenodecollection/index' doesn't exist. Creating new index...
> [junit4:junit4] 2> 16698 T2746 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362265676210/onenodecollection/index forceNew:false
> [junit4:junit4] 2> 16700 T2746 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@18b16779 lockFactory=org.apache.lucene.store.NativeFSLockFactory@e97411df),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4] 2> 16700 T2746 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
> [junit4:junit4] 2> 16701 T2746 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362265676210/onenodecollection/index
> [junit4:junit4] 2> 16703 T2746 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4:junit4] 2> 16704 T2746 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4:junit4] 2> 16705 T2746 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4:junit4] 2> 16706 T2746 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4:junit4] 2> 16707 T2746 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4:junit4] 2> 16707 T2746 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4:junit4] 2> 16708 T2746 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4:junit4] 2> 16709 T2746 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4:junit4] 2> 16710 T2746 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4:junit4] 2> 16711 T2746 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4:junit4] 2> 16713 T2746 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362265676210/onenodecollection
> [junit4:junit4] 2> 16714 T2746 oass.SolrIndexSearcher.<init> Opening Searcher@7559b3d3 main
> [junit4:junit4] 2> 16715 T2746 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362265676210/onenodecollection/tlog
> [junit4:junit4] 2> 16716 T2746 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4:junit4] 2> 16717 T2746 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4:junit4] 2> 16722 T2787 oasc.SolrCore.registerSearcher [onenodecollectioncore] Registered new searcher Searcher@7559b3d3 main{StandardDirectoryReader(segments_1:1)}
> [junit4:junit4] 2> 16725 T2746 oasc.ZkController.publish publishing core=onenodecollectioncore state=down
> [junit4:junit4] 2> 16950 T2731 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 16951 T2731 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:32892/jz/n",
> [junit4:junit4] 2> "state":"recovering",
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"4",
> [junit4:junit4] 2> "node_name":"127.0.0.1:32892_jz%2Fn",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard":"shard1",
> [junit4:junit4] 2> "core":"collection1"}
> [junit4:junit4] 2> 16955 T2731 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:60021/jz/n",
> [junit4:junit4] 2> "state":"down",
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":null,
> [junit4:junit4] 2> "node_name":"127.0.0.1:60021_jz%2Fn",
> [junit4:junit4] 2> "numShards":"1",
> [junit4:junit4] 2> "roles":"none",
> [junit4:junit4] 2> "collection":"onenodecollection",
> [junit4:junit4] 2> "shard":null,
> [junit4:junit4] 2> "core":"onenodecollectioncore"}
> [junit4:junit4] 2> 16956 T2731 oasc.Overseer$ClusterStateUpdater.createCollection Create collection onenodecollection with numShards 1
> [junit4:junit4] 2> 16956 T2731 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
> [junit4:junit4] 2> 16961 T2737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 16961 T2750 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 16962 T2766 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 16962 T2730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 16963 T2782 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 17727 T2746 oasc.CoreContainer.registerCore registering core: onenodecollectioncore
> [junit4:junit4] 2> 17728 T2746 oasc.ZkController.register Register replica - core:onenodecollectioncore address:http://127.0.0.1:60021/jz/n collection:onenodecollection shard:shard1
> [junit4:junit4] 2> 17729 T2746 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection/leader_elect/shard1/election
> [junit4:junit4] 2> 17735 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70006 type:delete cxid:0x9f zxid:0xac txntype:-1 reqpath:n/a Error Path:/solr/collections/onenodecollection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/onenodecollection/leaders
> [junit4:junit4] 2> 17737 T2746 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
> [junit4:junit4] 2> 17737 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70006 type:create cxid:0xa0 zxid:0xad txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 17740 T2746 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
> [junit4:junit4] 2> 17741 T2746 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
> [junit4:junit4] 2> 17742 T2746 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:60021/jz/n/onenodecollectioncore/
> [junit4:junit4] 2> 17742 T2746 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
> [junit4:junit4] 2> 17743 T2746 oasc.SyncStrategy.syncToMe http://127.0.0.1:60021/jz/n/onenodecollectioncore/ has no replicas
> [junit4:junit4] 2> 17744 T2746 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:60021/jz/n/onenodecollectioncore/
> [junit4:junit4] 2> 17744 T2746 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection/leaders/shard1
> [junit4:junit4] 2> 17748 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70006 type:create cxid:0xaa zxid:0xb1 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 18469 T2731 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 18474 T2750 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 18474 T2737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 18474 T2766 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 18474 T2730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 18474 T2782 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 18500 T2746 oasc.ZkController.register We are http://127.0.0.1:60021/jz/n/onenodecollectioncore/ and leader is http://127.0.0.1:60021/jz/n/onenodecollectioncore/
> [junit4:junit4] 2> 18501 T2746 oasc.ZkController.register No LogReplay needed for core=onenodecollectioncore baseURL=http://127.0.0.1:60021/jz/n
> [junit4:junit4] 2> 18502 T2746 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
> [junit4:junit4] 2> 18503 T2746 oasc.ZkController.publish publishing core=onenodecollectioncore state=active
> [junit4:junit4] 2> 18504 T2746 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 18505 T2708 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: onenodecollection failOnTimeout:true timeout (sec):230
> [junit4:junit4] 2> 18506 T2708 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> C176_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:32892/jz/n, state=recovering, node_name=127.0.0.1:32892_jz%2Fn, collection=collection1, shard=shard1, core=collection1}
> [junit4:junit4] 2> 19156 T2785 C176 P32892 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:60021/jz/n/collection1/ core=collection1 - recoveringAfterStartup=true
> [junit4:junit4] 2> 19157 T2785 C176 P32892 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:32892/jz/n START replicas=[http://127.0.0.1:60021/jz/n/collection1/] nUpdates=100
> [junit4:junit4] 2> 19157 T2785 C176 P32892 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
> [junit4:junit4] 2> 19158 T2785 C176 P32892 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
> [junit4:junit4] 2> 19158 T2785 C176 P32892 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
> [junit4:junit4] 2> 19158 T2785 C176 P32892 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
> [junit4:junit4] 2> 19159 T2785 C176 P32892 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
> [junit4:junit4] 2> 19159 T2785 C176 P32892 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:60021/jz/n/collection1/. core=collection1
> [junit4:junit4] 2> 19159 T2785 C176 P32892 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> ASYNC NEW_CORE C177 name=collection1 org.apache.solr.core.SolrCore@5465695e url=http://127.0.0.1:60021/jz/n/collection1 node=127.0.0.1:60021_jz%2Fn C177_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:60021/jz/n, state=active, node_name=127.0.0.1:60021_jz%2Fn, collection=collection1, shard=shard1, core=collection1, leader=true}
> [junit4:junit4] 2> 19353 T2744 C177 P60021 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 19354 T2743 C177 P60021 REQ /get {getVersions=100&qt=/get&distrib=false&version=2&wt=javabin} status=0 QTime=0
> [junit4:junit4] 2> 19354 T2744 C177 P60021 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1
> [junit4:junit4] 2> 19356 T2744 C177 P60021 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@c85e682f lockFactory=org.apache.lucene.store.NativeFSLockFactory@b9f9f41b),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4] 2> 19357 T2744 C177 P60021 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
> [junit4:junit4] 2> 19358 T2744 C177 P60021 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@c85e682f lockFactory=org.apache.lucene.store.NativeFSLockFactory@b9f9f41b),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@c85e682f lockFactory=org.apache.lucene.store.NativeFSLockFactory@b9f9f41b),segFN=segments_2,generation=2,filenames=[segments_2]
> [junit4:junit4] 2> 19358 T2744 C177 P60021 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
> [junit4:junit4] 2> 19359 T2744 C177 P60021 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1
> [junit4:junit4] 2> 19372 T2744 C177 P60021 oass.SolrIndexSearcher.<init> Opening Searcher@25753d95 realtime
> [junit4:junit4] 2> 19373 T2744 C177 P60021 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 19375 T2744 C177 P60021 /update {softCommit=false&commit_end_point=true&commit=true&version=2&waitSearcher=true&wt=javabin&openSearcher=false} {commit=} 0 22
> [junit4:junit4] 2> 19376 T2785 C176 P32892 oash.ReplicationHandler.inform Commits will be reserved for 10000
> [junit4:junit4] 2> 19376 T2785 C176 P32892 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
> [junit4:junit4] 2> 19378 T2743 C177 P60021 oash.ReplicationHandler.inform Commits will be reserved for 10000
> [junit4:junit4] 2> 19379 T2743 C177 P60021 REQ /replication {qt=/replication&version=2&command=indexversion&wt=javabin} status=0 QTime=2
> [junit4:junit4] 2> 19380 T2785 C176 P32892 oash.SnapPuller.fetchLatestIndex Master's generation: 2
> [junit4:junit4] 2> 19380 T2785 C176 P32892 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
> [junit4:junit4] 2> 19380 T2785 C176 P32892 oash.SnapPuller.fetchLatestIndex Starting replication process
> [junit4:junit4] 2> 19382 T2746 C177 P60021 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1
> [junit4:junit4] 2> 19383 T2746 C177 P60021 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1/index
> [junit4:junit4] 2> 19384 T2746 C177 P60021 REQ /replication {qt=/replication&version=2&generation=2&command=filelist&wt=javabin} status=0 QTime=2
> [junit4:junit4] 2> 19385 T2785 C176 P32892 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
> [junit4:junit4] 2> 19386 T2785 C176 P32892 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3/index.20130302200815591 forceNew:false
> [junit4:junit4] 2> 19387 T2785 C176 P32892 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3
> [junit4:junit4] 2> 19387 T2785 C176 P32892 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@48b5c8c0 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4a1bdf41) fullCopy=false
> [junit4:junit4] 2> 19389 T2744 C177 P60021 REQ /replication {checksum=true&qt=/replication&generation=2&command=filecontent&file=segments_2&wt=filestream} status=0 QTime=0
> [junit4:junit4] 2> 19390 T2785 C176 P32892 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3/index
> [junit4:junit4] 2> 19391 T2785 C176 P32892 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
> [junit4:junit4] 2> 19391 T2785 C176 P32892 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3
> [junit4:junit4] 2> 19392 T2785 C176 P32892 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3
> [junit4:junit4] 2> 19392 T2785 C176 P32892 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
> [junit4:junit4] 2> 19392 T2785 C176 P32892 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
> [junit4:junit4] 2> 19392 T2785 C176 P32892 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3
> [junit4:junit4] 2> 19393 T2785 C176 P32892 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=2
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@17091b05 lockFactory=org.apache.lucene.store.NativeFSLockFactory@58c80795),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@17091b05 lockFactory=org.apache.lucene.store.NativeFSLockFactory@58c80795),segFN=segments_2,generation=2,filenames=[segments_2]
> [junit4:junit4] 2> 19394 T2785 C176 P32892 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
> [junit4:junit4] 2> 19394 T2785 C176 P32892 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
> [junit4:junit4] 2> 19394 T2785 C176 P32892 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3
> [junit4:junit4] 2> 19395 T2785 C176 P32892 oass.SolrIndexSearcher.<init> Opening Searcher@b5797695 main
> [junit4:junit4] 2> 19396 T2784 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@b5797695 main{StandardDirectoryReader(segments_2:1:nrt)}
> [junit4:junit4] 2> 19396 T2784 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3/index
> [junit4:junit4] 2> 19396 T2785 C176 P32892 oash.SnapPuller.fetchLatestIndex removing temporary index download directory files MockDirWrapper(org.apache.lucene.store.RAMDirectory@48b5c8c0 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4a1bdf41)
> [junit4:junit4] 2> 19397 T2785 C176 P32892 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3/index.20130302200815591
> [junit4:junit4] 2> 19397 T2785 C176 P32892 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3/index
> [junit4:junit4] 2> 19397 T2785 C176 P32892 oasc.RecoveryStrategy.replay No replay needed. core=collection1
> [junit4:junit4] 2> 19397 T2785 C176 P32892 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
> [junit4:junit4] 2> 19398 T2785 C176 P32892 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4:junit4] 2> 19398 T2785 C176 P32892 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 19399 T2785 C176 P32892 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
> [junit4:junit4] 2> 19508 T2708 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 19979 T2731 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 19980 T2731 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:60021/jz/n",
> [junit4:junit4] 2> "state":"active",
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":null,
> [junit4:junit4] 2> "node_name":"127.0.0.1:60021_jz%2Fn",
> [junit4:junit4] 2> "numShards":"1",
> [junit4:junit4] 2> "roles":"none",
> [junit4:junit4] 2> "collection":"onenodecollection",
> [junit4:junit4] 2> "shard":"shard1",
> [junit4:junit4] 2> "core":"onenodecollectioncore"}
> [junit4:junit4] 2> 19983 T2731 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:32892/jz/n",
> [junit4:junit4] 2> "state":"active",
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"4",
> [junit4:junit4] 2> "node_name":"127.0.0.1:32892_jz%2Fn",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard":"shard1",
> [junit4:junit4] 2> "core":"collection1"}
> [junit4:junit4] 2> 19986 T2737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 19986 T2782 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 19986 T2750 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 19986 T2730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 19986 T2766 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 20510 T2708 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 20511 T2708 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: onenodecollection
> [junit4:junit4] 2> 20512 T2708 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> C176_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:32892/jz/n, state=active, node_name=127.0.0.1:32892_jz%2Fn, collection=collection1, shard=shard1, core=collection1}
> [junit4:junit4] 2> 20547 T2772 C176 P32892 /update {update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:60021/jz/n/collection1/&version=2&wt=javabin} {add=[1 (1428439115231133696)]} 0 1
> [junit4:junit4] 2> 20548 T2745 C177 P60021 /update {version=2&wt=javabin} {add=[1 (1428439115231133696)]} 0 7
> [junit4:junit4] 2> 20669 T2746 C177 P60021 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 20671 T2746 C177 P60021 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@c85e682f lockFactory=org.apache.lucene.store.NativeFSLockFactory@b9f9f41b),segFN=segments_2,generation=2,filenames=[segments_2]
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@c85e682f lockFactory=org.apache.lucene.store.NativeFSLockFactory@b9f9f41b),segFN=segments_3,generation=3,filenames=[_0_MockVariableIntBlock_0.tib, _0_Asserting_0.dvd, _0.fnm, _0.nvm, _0_MockVariableIntBlock_0.pos, _0_MockVariableIntBlock_0.tii, _0_Asserting_0.dvm, _0_MockVariableIntBlock_0.skp, _0_MockVariableIntBlock_0.doc, _0_Direct_0.pos, _0_Memory_0.ram, _0_Memory_1.ram, _0_Direct_0.tim, _0_Direct_0.doc, _0.si, segments_3, _0_Direct_0.tip, _0_MockVariableIntBlock_0.pyl, _0.fdt, _0_MockVariableIntBlock_0.frq, _0.fdx, _0.nvd]
> [junit4:junit4] 2> 20672 T2746 C177 P60021 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[_0_MockVariableIntBlock_0.tib, _0_Asserting_0.dvd, _0.fnm, _0.nvm, _0_MockVariableIntBlock_0.pos, _0_MockVariableIntBlock_0.tii, _0_Asserting_0.dvm, _0_MockVariableIntBlock_0.skp, _0_MockVariableIntBlock_0.doc, _0_Direct_0.pos, _0_Memory_0.ram, _0_Memory_1.ram, _0_Direct_0.tim, _0_Direct_0.doc, _0.si, segments_3, _0_Direct_0.tip, _0_MockVariableIntBlock_0.pyl, _0.fdt, _0_MockVariableIntBlock_0.frq, _0.fdx, _0.nvd]
> [junit4:junit4] 2> 20673 T2746 C177 P60021 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1
> [junit4:junit4] 2> 20674 T2746 C177 P60021 oass.SolrIndexSearcher.<init> Opening Searcher@ffd40ae5 main
> [junit4:junit4] 2> 20675 T2746 C177 P60021 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1/index
> [junit4:junit4] 2> 20676 T2746 C177 P60021 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 20677 T2752 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@ffd40ae5 main{StandardDirectoryReader(segments_3:3:nrt _0(5.0):C1)}
> [junit4:junit4] 2> 20678 T2752 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1/index
> [junit4:junit4] 2> 20678 T2746 C177 P60021 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:32892/jz/n/collection1/, StdNode: http://127.0.0.1:49462/jz/n/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
> [junit4:junit4] 2> 20680 T2773 C176 P32892 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 20682 T2773 C176 P32892 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@17091b05 lockFactory=org.apache.lucene.store.NativeFSLockFactory@58c80795),segFN=segments_2,generation=2,filenames=[segments_2]
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@17091b05 lockFactory=org.apache.lucene.store.NativeFSLockFactory@58c80795),segFN=segments_3,generation=3,filenames=[_0_MockVariableIntBlock_0.tib, _0_Asserting_0.dvd, _0.fnm, _0.nvm, _0_MockVariableIntBlock_0.pos, _0_MockVariableIntBlock_0.tii, _0_Asserting_0.dvm, _0_MockVariableIntBlock_0.skp, _0_MockVariableIntBlock_0.doc, _0_Direct_0.pos, _0_Memory_0.ram, _0_Memory_1.ram, _0_Direct_0.tim, _0_Direct_0.doc, _0.si, segments_3, _0_Direct_0.tip, _0_MockVariableIntBlock_0.pyl, _0.fdt, _0_MockVariableIntBlock_0.frq, _0.fdx, _0.nvd]
> [junit4:junit4] 2> 20683 T2773 C176 P32892 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[_0_MockVariableIntBlock_0.tib, _0_Asserting_0.dvd, _0.fnm, _0.nvm, _0_MockVariableIntBlock_0.pos, _0_MockVariableIntBlock_0.tii, _0_Asserting_0.dvm, _0_MockVariableIntBlock_0.skp, _0_MockVariableIntBlock_0.doc, _0_Direct_0.pos, _0_Memory_0.ram, _0_Memory_1.ram, _0_Direct_0.tim, _0_Direct_0.doc, _0.si, segments_3, _0_Direct_0.tip, _0_MockVariableIntBlock_0.pyl, _0.fdt, _0_MockVariableIntBlock_0.frq, _0.fdx, _0.nvd]
> [junit4:junit4] 2> ASYNC NEW_CORE C178 name=collection1 org.apache.solr.core.SolrCore@cf8d8b14 url=http://127.0.0.1:49462/jz/n/collection1 node=127.0.0.1:49462_jz%2Fn C178_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:49462/jz/n, state=active, node_name=127.0.0.1:49462_jz%2Fn, collection=collection1, shard=shard2, core=collection1, leader=true}
> [junit4:junit4] 2> 20683 T2759 C178 P49462 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
> [junit4:junit4] 2> 20684 T2773 C176 P32892 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3
> [junit4:junit4] 2> 20685 T2759 C178 P49462 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty2
> [junit4:junit4] 2> 20686 T2773 C176 P32892 oass.SolrIndexSearcher.<init> Opening Searcher@c6fdba14 main
> [junit4:junit4] 2> 20687 T2759 C178 P49462 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@e8168904 lockFactory=org.apache.lucene.store.NativeFSLockFactory@87db2c99),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4] 2> 20687 T2773 C176 P32892 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 20687 T2784 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@c6fdba14 main{StandardDirectoryReader(segments_3:3:nrt _0(5.0):C1)}
> [junit4:junit4] 2> 20687 T2759 C178 P49462 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
> [junit4:junit4] 2> 20689 T2784 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3/index
> [junit4:junit4] 2> 20689 T2759 C178 P49462 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@e8168904 lockFactory=org.apache.lucene.store.NativeFSLockFactory@87db2c99),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@e8168904 lockFactory=org.apache.lucene.store.NativeFSLockFactory@87db2c99),segFN=segments_2,generation=2,filenames=[segments_2]
> [junit4:junit4] 2> 20689 T2773 C176 P32892 /update {softCommit=false&expungeDeletes=false&commit_end_point=true&commit=true&version=2&waitSearcher=true&wt=javabin} {commit=} 0 9
> [junit4:junit4] 2> 20690 T2759 C178 P49462 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
> [junit4:junit4] 2> 20691 T2759 C178 P49462 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty2
> [junit4:junit4] 2> 20692 T2759 C178 P49462 oass.SolrIndexSearcher.<init> Opening Searcher@cced56cf main
> [junit4:junit4] 2> 20693 T2759 C178 P49462 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4] 2> 20693 T2768 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@cced56cf main{StandardDirectoryReader(segments_1:1)}
> [junit4:junit4] 2> 20695 T2768 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty2/index
> [junit4:junit4] 2> 20695 T2759 C178 P49462 /update {softCommit=false&expungeDeletes=false&commit_end_point=true&commit=true&version=2&waitSearcher=true&wt=javabin} {commit=} 0 12
> [junit4:junit4] 2> 20696 T2746 C177 P60021 /update {softCommit=false&commit=true&version=2&waitSearcher=true&wt=javabin} {commit=} 0 27
> [junit4:junit4] 2> 20702 T2759 C178 P49462 REQ /select {q=*:*&start=0&isShard=true&distrib=false&NOW=1362265696904&fl=id,score&version=2&fsv=true&rows=10&wt=javabin&shard.url=127.0.0.1:49462/jz/n/collection1/} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 20702 T2776 C176 P32892 REQ /select {q=*:*&start=0&isShard=true&distrib=false&NOW=1362265696904&fl=id,score&version=2&fsv=true&rows=10&wt=javabin&shard.url=127.0.0.1:60021/jz/n/collection1/|127.0.0.1:32892/jz/n/collection1/} hits=1 status=0 QTime=0
> [junit4:junit4] 2> 20705 T2775 C176 P32892 REQ /select {q=*:*&isShard=true&distrib=false&NOW=1362265696904&version=2&ids=1&wt=javabin&shard.url=127.0.0.1:60021/jz/n/collection1/|127.0.0.1:32892/jz/n/collection1/} status=0 QTime=0
> [junit4:junit4] 2> 20706 T2745 C177 P60021 REQ /select {q=*:*&version=2&wt=javabin} hits=1 status=0 QTime=7
> [junit4:junit4] 2> 20708 T2708 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> ASYNC NEW_CORE C179 name=onenodecollectioncore org.apache.solr.core.SolrCore@ffa0d4fa url=http://127.0.0.1:60021/jz/n/onenodecollectioncore node=127.0.0.1:60021_jz%2Fn C179_STATE=coll:onenodecollection core:onenodecollectioncore props:{base_url=http://127.0.0.1:60021/jz/n, state=active, node_name=127.0.0.1:60021_jz%2Fn, roles=none, collection=onenodecollection, shard=shard1, core=onenodecollectioncore, leader=true}
> [junit4:junit4] 2> 20717 T2743 C179 P60021 REQ /select {q=*:*&version=2&wt=javabin} hits=0 status=0 QTime=0
> [junit4:junit4] 2> 20719 T2708 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54711 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@bafd331c
> [junit4:junit4] 2> 20720 T2708 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4] 2> 20720 T2795 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54711. Will not attempt to authenticate using SASL (Access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4] 2> 20721 T2710 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:54958
> [junit4:junit4] 2> 21193 T2795 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54711, initiating session
> [junit4:junit4] 2> 21497 T2710 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:54958
> [junit4:junit4] 2> 21498 T2712 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d2d5c99b7000b with negotiated timeout 10000 for client /127.0.0.1:54958
> [junit4:junit4] 2> 21498 T2795 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54711, sessionid = 0x13d2d5c99b7000b, negotiated timeout = 10000
> [junit4:junit4] 2> 21498 T2796 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@bafd331c name:ZooKeeperConnection Watcher:127.0.0.1:54711 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4] 2> 21499 T2708 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4] 2> 21558 T2713 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d2d5c99b7000b
> [junit4:junit4] 2> 21560 T2710 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:54958 which had sessionid 0x13d2d5c99b7000b
> [junit4:junit4] 2> 21560 T2796 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4] 2> 21560 T2708 oaz.ZooKeeper.close Session: 0x13d2d5c99b7000b closed
> [junit4:junit4] 2> 21561 T2713 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d2d5c99b70004
> [junit4:junit4] 2> 21562 T2710 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:54928 which had sessionid 0x13d2d5c99b70004
> [junit4:junit4] 2> 21562 T2737 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4] 2> 21562 T2708 oaz.ZooKeeper.close Session: 0x13d2d5c99b70004 closed
> [junit4:junit4] 2> 21566 T2708 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 34136
> [junit4:junit4] 2> 21566 T2708 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1112915197
> [junit4:junit4] 2> 23000 T2731 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 23001 T2731 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:34136/jz/n",
> [junit4:junit4] 2> "state":"down",
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"1",
> [junit4:junit4] 2> "node_name":"127.0.0.1:34136_jz%2Fn",
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "collection":"control_collection",
> [junit4:junit4] 2> "shard":"shard1",
> [junit4:junit4] 2> "core":"collection1"}
> [junit4:junit4] 2> 23004 T2730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 23005 T2782 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 23004 T2750 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 23004 T2766 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 24568 T2708 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4] 2> 24569 T2708 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 24572 T2708 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@d99a2806
> [junit4:junit4] 2> 24575 T2708 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3/index
> [junit4:junit4] 2> 24577 T2708 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3
> [junit4:junit4] 2> 24579 T2708 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
> [junit4:junit4] 2> 24580 T2708 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
> [junit4:junit4] 2> 24580 T2708 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
> [junit4:junit4] 2> 24581 T2708 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
> [junit4:junit4] 2> 24582 T2708 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
> [junit4:junit4] 2> 24583 T2708 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/control/data/index
> [junit4:junit4] 2> 24584 T2708 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/control/data
> [junit4:junit4] 2> 24585 T2708 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/control/data
> [junit4:junit4] 2> 24586 T2708 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/control/data/index
> [junit4:junit4] 2> 24586 T2708 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/control/data/index
> [junit4:junit4] 2> 24588 T2731 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89277443356426243-127.0.0.1:34136_jz%2Fn-n_0000000000) am no longer a leader.
> [junit4:junit4] 2> 24588 T2713 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d2d5c99b70003
> [junit4:junit4] 2> 24589 T2710 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:54923 which had sessionid 0x13d2d5c99b70003
> [junit4:junit4] 2> 24590 T2708 oaz.ZooKeeper.close Session: 0x13d2d5c99b70003 closed
> [junit4:junit4] 2> 24590 T2766 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4:junit4] 2> 24590 T2782 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4:junit4] 2> 24591 T2766 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 24591 T2782 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 24590 T2750 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4:junit4] 2> 24592 T2750 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 24595 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70006 type:delete cxid:0xba zxid:0xcd txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
> [junit4:junit4] 2> 24595 T2750 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
> [junit4:junit4] 2> 24597 T2750 oasc.Overseer.start Overseer (id=89277443356426246-127.0.0.1:60021_jz%2Fn-n_0000000001) starting
> [junit4:junit4] 2> 24598 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70006 type:create cxid:0xbf zxid:0xcf txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 24598 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70006 type:create cxid:0xc0 zxid:0xd0 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 24599 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70006 type:create cxid:0xc1 zxid:0xd1 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 24600 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70006 type:create cxid:0xc2 zxid:0xd2 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 24602 T2798 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
> [junit4:junit4] 2> 24604 T2797 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
> [junit4:junit4] 2> 24607 T2797 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 24607 T2797 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:34136/jz/n",
> [junit4:junit4] 2> "state":"down",
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"1",
> [junit4:junit4] 2> "node_name":"127.0.0.1:34136_jz%2Fn",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "collection":"control_collection",
> [junit4:junit4] 2> "shard":"shard1",
> [junit4:junit4] 2> "core":"collection1"}
> [junit4:junit4] 2> 24613 T2766 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 24613 T2782 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 24614 T2750 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 24614 T2708 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/jz/n,null}
> [junit4:junit4] 2> 24667 T2708 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 60021
> [junit4:junit4] 2> 24667 T2708 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=-1556019191
> [junit4:junit4] 2> 25890 T2730 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
> [junit4:junit4] 2> 25890 T2730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4] 2> 25890 T2730 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
> [junit4:junit4] 2> 25891 T2730 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4] 2> 26117 T2797 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 26118 T2797 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:60021/jz/n",
> [junit4:junit4] 2> "state":"down",
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"127.0.0.1:60021_jz%2Fn_onenodecollectioncore",
> [junit4:junit4] 2> "node_name":"127.0.0.1:60021_jz%2Fn",
> [junit4:junit4] 2> "roles":"none",
> [junit4:junit4] 2> "collection":"onenodecollection",
> [junit4:junit4] 2> "shard":"shard1",
> [junit4:junit4] 2> "core":"onenodecollectioncore"}
> [junit4:junit4] 2> 26121 T2797 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:60021/jz/n",
> [junit4:junit4] 2> "state":"down",
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"2",
> [junit4:junit4] 2> "node_name":"127.0.0.1:60021_jz%2Fn",
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard":"shard1",
> [junit4:junit4] 2> "core":"collection1"}
> [junit4:junit4] 2> 26125 T2766 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 26125 T2782 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 26125 T2750 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 27673 T2708 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4] 2> 27673 T2708 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 27675 T2708 oasc.ZkController.publish publishing core=onenodecollectioncore state=down
> [junit4:junit4] 2> 27678 T2708 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@5465695e
> [junit4:junit4] 2> 27679 T2708 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3/index
> [junit4:junit4] 2> 27680 T2708 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3
> [junit4:junit4] 2> 27682 T2708 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=2,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=1,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
> [junit4:junit4] 2> 27683 T2708 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
> [junit4:junit4] 2> 27683 T2708 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
> [junit4:junit4] 2> 27684 T2708 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
> [junit4:junit4] 2> C177_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:60021/jz/n, state=down, node_name=127.0.0.1:60021_jz%2Fn, collection=collection1, shard=shard1, core=collection1, leader=true}
> [junit4:junit4] 2> 27685 T2708 C177 P60021 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1/index
> [junit4:junit4] 2> 27685 T2708 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
> [junit4:junit4] 2> 27686 T2708 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1/index
> [junit4:junit4] 2> 27687 T2708 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1/index
> [junit4:junit4] 2> 27687 T2708 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1/index
> [junit4:junit4] 2> 27688 T2708 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1
> [junit4:junit4] 2> 27688 T2708 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1
> [junit4:junit4] 2> 27689 T2708 oasc.SolrCore.close [onenodecollectioncore] CLOSING SolrCore org.apache.solr.core.SolrCore@ffa0d4fa
> [junit4:junit4] 2> 27694 T2708 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
> [junit4:junit4] 2> 27695 T2708 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
> [junit4:junit4] 2> 27696 T2708 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
> [junit4:junit4] 2> 27697 T2708 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
> [junit4:junit4] 2> 27697 T2708 oasc.SolrCore.closeSearcher [onenodecollectioncore] Closing main searcher on request.
> [junit4:junit4] 2> 27698 T2708 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362265676210/onenodecollection/index
> [junit4:junit4] 2> 27699 T2708 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362265676210/onenodecollection
> [junit4:junit4] 2> 27699 T2708 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362265676210/onenodecollection
> [junit4:junit4] 2> 27700 T2708 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362265676210/onenodecollection/index
> [junit4:junit4] 2> 27700 T2708 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362265676210/onenodecollection/index
> [junit4:junit4] 2> 27702 T2797 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89277443356426246-127.0.0.1:60021_jz%2Fn-n_0000000001) am no longer a leader.
> [junit4:junit4] 2> 27702 T2713 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d2d5c99b70006
> [junit4:junit4] 2> 27703 T2710 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:54930 which had sessionid 0x13d2d5c99b70006
> [junit4:junit4] 2> 27704 T2750 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
> [junit4:junit4] 2> 27704 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b7000a type:delete cxid:0x53 zxid:0xe3 txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard1 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard1
> [junit4:junit4] 2> 27703 T2708 oaz.ZooKeeper.close Session: 0x13d2d5c99b70006 closed
> [junit4:junit4] 2> 27704 T2750 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
> [junit4:junit4] 2> 27705 T2782 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
> [junit4:junit4] 2> 27704 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70008 type:delete cxid:0x6e zxid:0xe4 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
> [junit4:junit4] 2> 27705 T2750 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
> [junit4:junit4] 2> 27705 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b7000a type:create cxid:0x54 zxid:0xe5 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 27706 T2750 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4] 2> 27706 T2766 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
> [junit4:junit4] 2> 27707 T2782 oasc.ShardLeaderElectionContext.shouldIBeLeader Checking if I should try and be the leader.
> [junit4:junit4] 2> 27708 T2782 oasc.ShardLeaderElectionContext.shouldIBeLeader My last published State was Active, it's okay to be the leader.
> [junit4:junit4] 2> 27708 T2782 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
> [junit4:junit4] 2> 27708 T2782 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:32892/jz/n/collection1/
> [junit4:junit4] 2> 27709 T2766 oasc.Overseer.start Overseer (id=89277443356426248-127.0.0.1:49462_jz%2Fn-n_0000000002) starting
> [junit4:junit4] 2> 27709 T2782 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:32892/jz/n START replicas=[http://127.0.0.1:60021/jz/n/collection1/] nUpdates=100
> [junit4:junit4] 2> 27709 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70008 type:create cxid:0x73 zxid:0xe8 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 27710 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70008 type:create cxid:0x74 zxid:0xe9 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 27711 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70008 type:create cxid:0x75 zxid:0xea txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 27712 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70008 type:create cxid:0x76 zxid:0xeb txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 27712 T2801 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
> [junit4:junit4] 2> 27713 T2766 oascc.ZkStateReader$3.process Updating live nodes... (2)
> [junit4:junit4] 2> 27713 T2766 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 27715 T2800 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
> [junit4:junit4] 2> 27716 T2782 oasu.PeerSync.handleResponse WARNING PeerSync: core=collection1 url=http://127.0.0.1:32892/jz/n couldn't connect to http://127.0.0.1:60021/jz/n/collection1/, counting as success
> [junit4:junit4] 2> 27716 T2782 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:32892/jz/n DONE. sync succeeded
> [junit4:junit4] 2> 27716 T2782 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
> [junit4:junit4] 2> 27717 T2800 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
> [junit4:junit4] 2> 27717 T2782 oasc.SyncStrategy.syncToMe http://127.0.0.1:32892/jz/n/collection1/: try and ask http://127.0.0.1:60021/jz/n/collection1/ to sync
> [junit4:junit4] 2> 27717 T2800 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:60021/jz/n",
> [junit4:junit4] 2> "state":"down",
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":"2",
> [junit4:junit4] 2> "node_name":"127.0.0.1:60021_jz%2Fn",
> [junit4:junit4] 2> "numShards":"2",
> [junit4:junit4] 2> "roles":null,
> [junit4:junit4] 2> "collection":"collection1",
> [junit4:junit4] 2> "shard":"shard1",
> [junit4:junit4] 2> "core":"collection1"}
> [junit4:junit4] 2> 27718 T2782 oasc.SolrException.log SEVERE Sync request error: org.apache.solr.client.solrj.SolrServerException: Server refused connection at: http://127.0.0.1:60021/jz/n/collection1
> [junit4:junit4] 2> 27719 T2782 oasc.SyncStrategy.syncToMe http://127.0.0.1:32892/jz/n/collection1/: Sync failed - asking replica (http://127.0.0.1:60021/jz/n/collection1/) to recover.
> [junit4:junit4] 2> 27719 T2782 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:32892/jz/n/collection1/
> [junit4:junit4] 2> 27719 T2804 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4] 2> 27719 T2782 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
> [junit4:junit4] 2> 27723 T2800 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
> [junit4:junit4] 2> "base_url":"http://127.0.0.1:60021/jz/n",
> [junit4:junit4] 2> "state":"down",
> [junit4:junit4] 2> "operation":"state",
> [junit4:junit4] 2> "core_node_name":null,
> [junit4:junit4] 2> "node_name":"127.0.0.1:60021_jz%2Fn",
> [junit4:junit4] 2> "numShards":"1",
> [junit4:junit4] 2> "roles":"none",
> [junit4:junit4] 2> "collection":"onenodecollection",
> [junit4:junit4] 2> "shard":"shard1",
> [junit4:junit4] 2> "core":"onenodecollectioncore"}
> [junit4:junit4] 2> 27723 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b7000a type:create cxid:0x5c zxid:0xef txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 27725 T2782 oascc.ZkStateReader$3.process Updating live nodes... (2)
> [junit4:junit4] 2> 27725 T2782 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 27726 T2804 oasc.SolrException.log SEVERE http://127.0.0.1:32892/jz/n/collection1/: Could not tell a replica to recover:org.apache.solr.client.solrj.SolrServerException: Server refused connection at: http://127.0.0.1:60021/jz/n
> [junit4:junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:409)
> [junit4:junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181)
> [junit4:junit4] 2> at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:298)
> [junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
> [junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:614)
> [junit4:junit4] 2> at java.lang.Thread.run(Thread.java:780)
> [junit4:junit4] 2> Caused by: org.apache.http.conn.HttpHostConnectException: Connection to http://127.0.0.1:60021 refused
> [junit4:junit4] 2> at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:190)
> [junit4:junit4] 2> at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
> [junit4:junit4] 2> at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
> [junit4:junit4] 2> at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
> [junit4:junit4] 2> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
> [junit4:junit4] 2> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
> [junit4:junit4] 2> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
> [junit4:junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:353)
> [junit4:junit4] 2> ... 5 more
> [junit4:junit4] 2> Caused by: java.net.ConnectException: Connection refused
> [junit4:junit4] 2> at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:369)
> [junit4:junit4] 2> at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:230)
> [junit4:junit4] 2> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:212)
> [junit4:junit4] 2> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:402)
> [junit4:junit4] 2> at java.net.Socket.connect(Socket.java:590)
> [junit4:junit4] 2> at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
> [junit4:junit4] 2> at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
> [junit4:junit4] 2> ... 12 more
> [junit4:junit4] 2>
> [junit4:junit4] 2> 27726 T2708 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/jz/n,null}
> [junit4:junit4] 2> 27732 T2766 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 27732 T2782 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 27778 T2708 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 49462
> [junit4:junit4] 2> 27779 T2708 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1479167609
> [junit4:junit4] 2> 28781 T2708 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4] 2> 28782 T2708 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4] 2> 28785 T2708 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@cf8d8b14
> [junit4:junit4] 2> 28788 T2708 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3/index
> [junit4:junit4] 2> 28789 T2708 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3
> [junit4:junit4] 2> 28792 T2708 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
> [junit4:junit4] 2> 28793 T2708 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
> [junit4:junit4] 2> 28794 T2708 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
> [junit4:junit4] 2> 28795 T2708 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
> [junit4:junit4] 2> 28796 T2708 C178 P49462 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty2/index
> [junit4:junit4] 2> 28797 T2708 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
> [junit4:junit4] 2> 28798 T2708 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty2/index
> [junit4:junit4] 2> 28799 T2708 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty2/index
> [junit4:junit4] 2> 28800 T2708 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty2/index
> [junit4:junit4] 2> 28801 T2708 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty2
> [junit4:junit4] 2> 28801 T2708 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty2
> [junit4:junit4] 2> 28803 T2800 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89277443356426248-127.0.0.1:49462_jz%2Fn-n_0000000002) am no longer a leader.
> [junit4:junit4] 2> 28803 T2713 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d2d5c99b70008
> [junit4:junit4] 2> 28804 T2710 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:54937 which had sessionid 0x13d2d5c99b70008
> [junit4:junit4] 2> 28805 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b7000a type:delete cxid:0x64 zxid:0xff txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
> [junit4:junit4] 2> 28805 T2708 oaz.ZooKeeper.close Session: 0x13d2d5c99b70008 closed
> [junit4:junit4] 2> 28805 T2782 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
> [junit4:junit4] 2> 28807 T2782 oasc.Overseer.start Overseer (id=89277443356426250-127.0.0.1:32892_jz%2Fn-n_0000000003) starting
> [junit4:junit4] 2> 28808 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b7000a type:create cxid:0x69 zxid:0x101 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 28809 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b7000a type:create cxid:0x6a zxid:0x102 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 28809 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b7000a type:create cxid:0x6b zxid:0x103 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 28810 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b7000a type:create cxid:0x6c zxid:0x104 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4] 2> 28811 T2806 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
> [junit4:junit4] 2> 28812 T2782 oascc.ZkStateReader$3.process Updating live nodes... (1)
> [junit4:junit4] 2> 28812 T2782 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
> [junit4:junit4] 2> 28813 T2805 o
>
> [...truncated too long message...]
>
> nit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
> [junit4:junit4] > at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
> [junit4:junit4] > at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
> [junit4:junit4] > at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> [junit4:junit4] > at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
> [junit4:junit4] > at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
> [junit4:junit4] > at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
> [junit4:junit4] > at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
> [junit4:junit4] > at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> [junit4:junit4] > at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
> [junit4:junit4] > at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> [junit4:junit4] > at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
> [junit4:junit4] > at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> [junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
> [junit4:junit4] > at java.lang.Thread.run(Thread.java:780)
> [junit4:junit4] 2> 30025 T2708 oas.SolrTestCaseJ4.deleteCore ###deleteCore
> [junit4:junit4] 2> 30029 T2707 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 3 leaked thread(s).
> [junit4:junit4] 2> 30104 T2766 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
> [junit4:junit4] 2> 30104 T2766 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
> [junit4:junit4] 2> 30105 T2766 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
> [junit4:junit4] 2> 30105 T2766 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4] 2> 30795 T2711 oazs.SessionTrackerImpl.run SessionTrackerImpl exited loop!
> [junit4:junit4] 2> 31203 T2782 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
> [junit4:junit4] 2> 31203 T2782 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
> [junit4:junit4] 2> 31204 T2782 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
> [junit4:junit4] 2> 31205 T2782 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4] 2> NOTE: test params are: codec=Lucene42: {range_facet_si=PostingsFormat(name=Memory doPackFST= true), range_facet_l=PostingsFormat(name=Direct), intDefault=PostingsFormat(name=Direct), range_facet_sl=MockVariableIntBlock(baseBlockSize=114), timestamp=PostingsFormat(name=Direct), _version_=PostingsFormat(name=Direct), multiDefault=PostingsFormat(name=Memory doPackFST= true), id=PostingsFormat(name=Memory doPackFST= false)}, docValues:{timestamp=DocValuesFormat(name=Asserting)}, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=ms_MY, timezone=Brazil/East
> [junit4:junit4] 2> NOTE: Linux 3.2.0-38-generic x86/IBM Corporation 1.7.0 (32-bit)/cpus=8,threads=3,free=90293600,total=200278016
> [junit4:junit4] 2> NOTE: All tests run in this JVM: [BasicDistributedZkTest, TestSolrXMLSerializer, TestIndexSearcher, TestGroupingSearch, DOMUtilTest, TestIndexingPerformance, TestLMDirichletSimilarityFactory, SampleTest, SolrIndexConfigTest, TestSearchPerf, TestComponentsName, SpellingQueryConverterTest, TestElisionMultitermQuery, TestConfig, SolrCoreCheckLockOnStartupTest, TestReload, TestBinaryResponseWriter, TestDefaultSimilarityFactory, ShardRoutingCustomTest, TestPHPSerializedResponseWriter, TestStressReorder, TestArbitraryIndexDir, PrimUtilsTest, TestSolrIndexConfig, QueryElevationComponentTest, TestWordDelimiterFilterFactory, SolrCmdDistributorTest, SimplePostToolTest, TestPropInject, HardAutoCommitTest, OutputWriterTest, SpatialFilterTest, DebugComponentTest, LeaderElectionIntegrationTest, SpellCheckCollatorTest, TestOmitPositions, TestCodecSupport, DirectSolrConnectionTest, TestRTGBase, ReturnFieldsTest, FileUtilsTest, TestSurroundQueryParser, TestUpdate, TestBinaryField, IndexBasedSpellCheckerTest, SpellCheckComponentTest, XmlUpdateRequestHandlerTest, ScriptEngineTest, DistributedTermsComponentTest, ZkCLITest, LoggingHandlerTest, PluginInfoTest, TestDocumentBuilder, ClusterStateTest, PreAnalyzedFieldTest, ClusterStateUpdateTest, DocValuesMultiTest, WordBreakSolrSpellCheckerTest, TestSolrJ, DateMathParserTest, SpellPossibilityIteratorTest, TestSolr4Spatial, DefaultValueUpdateProcessorTest, OverseerTest, FullSolrCloudDistribCmdsTest, DistributedSpellCheckComponentTest, TestRealTimeGet, TestRandomFaceting, TestRangeQuery, DirectUpdateHandlerTest, SimpleFacetsTest, TestCoreContainer, LukeRequestHandlerTest, TestFiltering, FileBasedSpellCheckerTest, PeerSyncTest, BasicFunctionalityTest, TestTrie, CacheHeaderTest, IndexSchemaTest, TermsComponentTest, XsltUpdateRequestHandlerTest, PingRequestHandlerTest, DocumentAnalysisRequestHandlerTest, DirectUpdateHandlerOptimizeTest, TestValueSourceCache, JSONWriterTest, SearchHandlerTest, SuggesterFSTTest, SuggesterTest, ShowFileRequestHandlerTest, PrimitiveFieldTypeTest, FieldMutatingUpdateProcessorTest, TestSolrDeletionPolicy2, TestPropInjectDefaults, TestXIncludeConfig, TestLFUCache, TestQuerySenderListener, NumericFieldsTest, TestJmxMonitoredMap, TestBM25SimilarityFactory, TestPluginEnable, TestDFRSimilarityFactory, TimeZoneUtilsTest, OpenExchangeRatesOrgProviderTest, TestSystemIdResolver, TestBadConfig, RAMDirectoryFactoryTest, SystemInfoHandlerTest, TestUtils, AnalysisAfterCoreReloadTest, TestRandomDVFaceting, TestCharFilters, BasicDistributedZk2Test]
> [junit4:junit4] Completed on J0 in 31.30s, 1 test, 1 failure <<< FAILURES!
>
> [...truncated 81 lines...]
> BUILD FAILED
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:381: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:374: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:39: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:183: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:449: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1207: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:870: There were test failures: 260 suites, 1095 tests, 1 failure, 10 ignored (4 assumptions)
>
> Total time: 46 minutes 23 seconds
> Build step 'Invoke Ant' marked build as failure
> Archiving artifacts
> Recording test results
> Description set: Java: 32bit/ibm-j9-jdk7
> Email was triggered for: Failure
> Sending email for trigger: Failure
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[JENKINS] Lucene-Solr-trunk-Linux (32bit/ibm-j9-jdk7) - Build #
4543 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/4543/
Java: 32bit/ibm-j9-jdk7
1 tests failed.
FAILED: org.apache.solr.cloud.BasicDistributedZk2Test.testDistribSearch
Error Message:
expected:<1> but was:<0>
Stack Trace:
java.lang.AssertionError: expected:<1> but was:<0>
at __randomizedtesting.SeedInfo.seed([E534325F463786F:8FB5CD3D833C1853]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.BasicDistributedZk2Test.testNodeWithoutCollectionForwarding(BasicDistributedZk2Test.java:209)
at org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:89)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:88)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:55)
at java.lang.reflect.Method.invoke(Method.java:613)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:780)
Build Log:
[...truncated 9499 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZk2Test
[junit4:junit4] 2> 0 T2708 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /jz/n
[junit4:junit4] 2> 4 T2708 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362265676210
[junit4:junit4] 2> 5 T2708 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 6 T2709 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 8 T2709 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4] 2> 8 T2709 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4] 2> 9 T2709 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4] 2> 9 T2709 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4] 2> 10 T2709 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4] 2> 106 T2708 oasc.ZkTestServer.run start zk server on port:54711
[junit4:junit4] 2> 107 T2708 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54711 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@333af0cc
[junit4:junit4] 2> 108 T2714 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54711. Will not attempt to authenticate using SASL (Access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 108 T2708 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 108 T2714 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54711, initiating session
[junit4:junit4] 2> 108 T2710 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:54920
[junit4:junit4] 2> 109 T2710 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:54920
[junit4:junit4] 2> 109 T2712 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4] 2> 126 T2712 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d2d5c99b70000 with negotiated timeout 10000 for client /127.0.0.1:54920
[junit4:junit4] 2> 126 T2714 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54711, sessionid = 0x13d2d5c99b70000, negotiated timeout = 10000
[junit4:junit4] 2> 127 T2715 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@333af0cc name:ZooKeeperConnection Watcher:127.0.0.1:54711 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 127 T2708 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 128 T2708 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 130 T2713 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d2d5c99b70000
[junit4:junit4] 2> 131 T2715 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 131 T2710 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:54920 which had sessionid 0x13d2d5c99b70000
[junit4:junit4] 2> 131 T2708 oaz.ZooKeeper.close Session: 0x13d2d5c99b70000 closed
[junit4:junit4] 2> 132 T2708 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54711/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@6aa2ff8a
[junit4:junit4] 2> 133 T2716 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54711. Will not attempt to authenticate using SASL (Access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 133 T2708 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 133 T2710 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:54921
[junit4:junit4] 2> 133 T2716 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54711, initiating session
[junit4:junit4] 2> 134 T2710 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:54921
[junit4:junit4] 2> 134 T2712 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d2d5c99b70001 with negotiated timeout 10000 for client /127.0.0.1:54921
[junit4:junit4] 2> 135 T2716 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54711, sessionid = 0x13d2d5c99b70001, negotiated timeout = 10000
[junit4:junit4] 2> 135 T2717 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6aa2ff8a name:ZooKeeperConnection Watcher:127.0.0.1:54711/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 135 T2708 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 136 T2708 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 138 T2708 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 140 T2708 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 141 T2708 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 143 T2708 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 144 T2708 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 146 T2708 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 147 T2708 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 150 T2708 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 150 T2708 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 152 T2708 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 153 T2708 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 155 T2708 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 156 T2708 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 159 T2708 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 159 T2708 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 162 T2708 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 163 T2708 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 165 T2708 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 166 T2708 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 168 T2708 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 169 T2708 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 171 T2713 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d2d5c99b70001
[junit4:junit4] 2> 172 T2717 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 172 T2710 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:54921 which had sessionid 0x13d2d5c99b70001
[junit4:junit4] 2> 172 T2708 oaz.ZooKeeper.close Session: 0x13d2d5c99b70001 closed
[junit4:junit4] 2> 232 T2708 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 245 T2708 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:34136
[junit4:junit4] 2> 246 T2708 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 246 T2708 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 247 T2708 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362265676378
[junit4:junit4] 2> 247 T2708 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362265676378/solr.xml
[junit4:junit4] 2> 248 T2708 oasc.CoreContainer.<init> New CoreContainer 1112915197
[junit4:junit4] 2> 248 T2708 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362265676378/'
[junit4:junit4] 2> 249 T2708 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362265676378/'
[junit4:junit4] 2> 260 T2708 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 261 T2708 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 261 T2708 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 262 T2708 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 262 T2708 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 263 T2708 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 264 T2708 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 264 T2708 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 265 T2708 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 265 T2708 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 272 T2708 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 282 T2708 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:54711/solr
[junit4:junit4] 2> 282 T2708 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 283 T2708 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54711 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@6954bf70
[junit4:junit4] 2> 284 T2727 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54711. Will not attempt to authenticate using SASL (Access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 284 T2708 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 284 T2727 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54711, initiating session
[junit4:junit4] 2> 285 T2710 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:54922
[junit4:junit4] 2> 285 T2710 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:54922
[junit4:junit4] 2> 286 T2712 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d2d5c99b70002 with negotiated timeout 20000 for client /127.0.0.1:54922
[junit4:junit4] 2> 286 T2727 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54711, sessionid = 0x13d2d5c99b70002, negotiated timeout = 20000
[junit4:junit4] 2> 287 T2728 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6954bf70 name:ZooKeeperConnection Watcher:127.0.0.1:54711 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 287 T2708 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 288 T2713 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d2d5c99b70002
[junit4:junit4] 2> 289 T2728 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 289 T2710 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:54922 which had sessionid 0x13d2d5c99b70002
[junit4:junit4] 2> 289 T2708 oaz.ZooKeeper.close Session: 0x13d2d5c99b70002 closed
[junit4:junit4] 2> 290 T2708 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 294 T2708 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54711/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@cf20b951
[junit4:junit4] 2> 295 T2729 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54711. Will not attempt to authenticate using SASL (Access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 295 T2708 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 295 T2729 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54711, initiating session
[junit4:junit4] 2> 296 T2710 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:54923
[junit4:junit4] 2> 296 T2710 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:54923
[junit4:junit4] 2> 297 T2712 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d2d5c99b70003 with negotiated timeout 20000 for client /127.0.0.1:54923
[junit4:junit4] 2> 297 T2729 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54711, sessionid = 0x13d2d5c99b70003, negotiated timeout = 20000
[junit4:junit4] 2> 298 T2730 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@cf20b951 name:ZooKeeperConnection Watcher:127.0.0.1:54711/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 298 T2708 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 299 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 300 T2708 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 302 T2708 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34136_jz%2Fn
[junit4:junit4] 2> 303 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70003 type:delete cxid:0x8 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:34136_jz%2Fn Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:34136_jz%2Fn
[junit4:junit4] 2> 303 T2708 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:34136_jz%2Fn
[junit4:junit4] 2> 305 T2708 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 308 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70003 type:delete cxid:0x16 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4] 2> 308 T2708 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 311 T2708 oasc.Overseer.start Overseer (id=89277443356426243-127.0.0.1:34136_jz%2Fn-n_0000000000) starting
[junit4:junit4] 2> 312 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70003 type:create cxid:0x1b zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 313 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70003 type:create cxid:0x1c zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 313 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70003 type:create cxid:0x1d zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 314 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70003 type:create cxid:0x1e zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 315 T2732 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 315 T2708 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 317 T2708 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 318 T2731 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 320 T2733 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362265676378/collection1
[junit4:junit4] 2> 320 T2733 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 321 T2733 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 321 T2733 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 322 T2733 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362265676378/collection1/'
[junit4:junit4] 2> 322 T2733 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362265676378/collection1/lib/README' to classloader
[junit4:junit4] 2> 323 T2733 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362265676378/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 347 T2733 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 378 T2733 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 379 T2733 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 383 T2733 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 691 T2733 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 695 T2733 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 698 T2733 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 707 T2733 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 710 T2733 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 713 T2733 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 714 T2733 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 714 T2733 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 715 T2733 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 716 T2733 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 716 T2733 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 716 T2733 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1362265676378/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/control/data/
[junit4:junit4] 2> 717 T2733 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3de91d1e
[junit4:junit4] 2> 717 T2733 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 718 T2733 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/control/data forceNew:false
[junit4:junit4] 2> 718 T2733 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/control/data
[junit4:junit4] 2> 719 T2733 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/control/data/index/
[junit4:junit4] 2> 719 T2733 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 719 T2733 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/control/data/index forceNew:false
[junit4:junit4] 2> 721 T2733 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@beba4872 lockFactory=org.apache.lucene.store.NativeFSLockFactory@ca29a6eb),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 721 T2733 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 721 T2733 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/control/data/index
[junit4:junit4] 2> 723 T2733 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 723 T2733 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 723 T2733 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 724 T2733 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 724 T2733 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 725 T2733 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 725 T2733 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 725 T2733 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 726 T2733 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 727 T2733 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 728 T2733 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/control/data
[junit4:junit4] 2> 729 T2733 oass.SolrIndexSearcher.<init> Opening Searcher@2581835 main
[junit4:junit4] 2> 729 T2733 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/control/data/tlog
[junit4:junit4] 2> 730 T2733 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 730 T2733 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 734 T2734 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2581835 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 735 T2733 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 735 T2733 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 736 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70003 type:create cxid:0x50 zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4] 2> 1821 T2731 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1821 T2731 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34136/jz/n",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "node_name":"127.0.0.1:34136_jz%2Fn",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 1822 T2731 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 1822 T2731 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 1822 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70003 type:create cxid:0x58 zxid:0x2c txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4] 2> 1827 T2730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 2737 T2733 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 2738 T2733 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:34136/jz/n collection:control_collection shard:shard1
[junit4:junit4] 2> 2739 T2733 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 2743 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70003 type:delete cxid:0x72 zxid:0x36 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4] 2> 2744 T2733 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 2744 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70003 type:create cxid:0x73 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 2746 T2733 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 2746 T2733 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 2746 T2733 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:34136/jz/n/collection1/
[junit4:junit4] 2> 2747 T2733 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 2747 T2733 oasc.SyncStrategy.syncToMe http://127.0.0.1:34136/jz/n/collection1/ has no replicas
[junit4:junit4] 2> 2747 T2733 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:34136/jz/n/collection1/
[junit4:junit4] 2> 2748 T2733 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 2750 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70003 type:create cxid:0x7d zxid:0x3b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 3332 T2731 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 3343 T2730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 3353 T2733 oasc.ZkController.register We are http://127.0.0.1:34136/jz/n/collection1/ and leader is http://127.0.0.1:34136/jz/n/collection1/
[junit4:junit4] 2> 3353 T2733 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:34136/jz/n
[junit4:junit4] 2> 3354 T2733 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 3354 T2733 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 3355 T2733 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 3357 T2733 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 3359 T2708 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 3360 T2708 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 3362 T2708 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 3368 T2708 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 3369 T2708 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54711/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@30f40218
[junit4:junit4] 2> 3370 T2736 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54711. Will not attempt to authenticate using SASL (Access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 3370 T2708 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 3371 T2710 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:54928
[junit4:junit4] 2> 3370 T2736 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54711, initiating session
[junit4:junit4] 2> 3371 T2710 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:54928
[junit4:junit4] 2> 3372 T2712 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d2d5c99b70004 with negotiated timeout 10000 for client /127.0.0.1:54928
[junit4:junit4] 2> 3372 T2736 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54711, sessionid = 0x13d2d5c99b70004, negotiated timeout = 10000
[junit4:junit4] 2> 3373 T2737 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@30f40218 name:ZooKeeperConnection Watcher:127.0.0.1:54711/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 3373 T2708 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 3374 T2708 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 3375 T2708 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 3436 T2708 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 3438 T2708 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:60021
[junit4:junit4] 2> 3439 T2708 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 3440 T2708 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 3440 T2708 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362265679580
[junit4:junit4] 2> 3441 T2708 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362265679580/solr.xml
[junit4:junit4] 2> 3441 T2708 oasc.CoreContainer.<init> New CoreContainer -1556019191
[junit4:junit4] 2> 3442 T2708 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362265679580/'
[junit4:junit4] 2> 3442 T2708 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362265679580/'
[junit4:junit4] 2> 3455 T2708 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 3456 T2708 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 3456 T2708 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 3457 T2708 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 3457 T2708 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 3458 T2708 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 3458 T2708 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 3459 T2708 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 3459 T2708 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 3460 T2708 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 3467 T2708 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 3478 T2708 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:54711/solr
[junit4:junit4] 2> 3478 T2708 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 3479 T2708 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54711 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@73cd271d
[junit4:junit4] 2> 3480 T2747 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54711. Will not attempt to authenticate using SASL (Access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 3480 T2708 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 3480 T2747 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54711, initiating session
[junit4:junit4] 2> 3481 T2710 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:54929
[junit4:junit4] 2> 3481 T2710 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:54929
[junit4:junit4] 2> 3482 T2712 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d2d5c99b70005 with negotiated timeout 20000 for client /127.0.0.1:54929
[junit4:junit4] 2> 3482 T2747 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54711, sessionid = 0x13d2d5c99b70005, negotiated timeout = 20000
[junit4:junit4] 2> 3482 T2748 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@73cd271d name:ZooKeeperConnection Watcher:127.0.0.1:54711 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 3483 T2708 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 3484 T2713 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d2d5c99b70005
[junit4:junit4] 2> 3484 T2748 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 3484 T2710 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:54929 which had sessionid 0x13d2d5c99b70005
[junit4:junit4] 2> 3484 T2708 oaz.ZooKeeper.close Session: 0x13d2d5c99b70005 closed
[junit4:junit4] 2> 3485 T2708 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 3489 T2708 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54711/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@996c3c56
[junit4:junit4] 2> 3491 T2749 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54711. Will not attempt to authenticate using SASL (Access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 3490 T2708 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 3491 T2749 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54711, initiating session
[junit4:junit4] 2> 3491 T2710 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:54930
[junit4:junit4] 2> 3492 T2710 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:54930
[junit4:junit4] 2> 3492 T2712 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d2d5c99b70006 with negotiated timeout 20000 for client /127.0.0.1:54930
[junit4:junit4] 2> 3492 T2749 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54711, sessionid = 0x13d2d5c99b70006, negotiated timeout = 20000
[junit4:junit4] 2> 3493 T2750 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@996c3c56 name:ZooKeeperConnection Watcher:127.0.0.1:54711/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 3493 T2708 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 3494 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70006 type:create cxid:0x1 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 3495 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70006 type:create cxid:0x2 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 3496 T2708 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 4498 T2708 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60021_jz%2Fn
[junit4:junit4] 2> 4499 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70006 type:delete cxid:0x9 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:60021_jz%2Fn Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:60021_jz%2Fn
[junit4:junit4] 2> 4500 T2708 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:60021_jz%2Fn
[junit4:junit4] 2> 4502 T2737 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 4502 T2750 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 4502 T2730 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 4502 T2730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 4506 T2751 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362265679580/collection1
[junit4:junit4] 2> 4506 T2751 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 4507 T2751 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 4507 T2751 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 4508 T2751 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362265679580/collection1/'
[junit4:junit4] 2> 4508 T2751 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362265679580/collection1/lib/README' to classloader
[junit4:junit4] 2> 4509 T2751 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362265679580/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 4532 T2751 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 4565 T2751 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 4566 T2751 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 4570 T2751 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 4849 T2731 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 4850 T2731 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34136/jz/n",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "node_name":"127.0.0.1:34136_jz%2Fn",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 4852 T2730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 4852 T2750 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 4852 T2737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 4888 T2751 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 4893 T2751 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 4896 T2751 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 4906 T2751 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 4911 T2751 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 4916 T2751 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 4917 T2751 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 4918 T2751 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 4918 T2751 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 4920 T2751 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 4920 T2751 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 4921 T2751 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362265679580/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1/
[junit4:junit4] 2> 4921 T2751 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3de91d1e
[junit4:junit4] 2> 4922 T2751 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 4923 T2751 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1 forceNew:false
[junit4:junit4] 2> 4923 T2751 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1
[junit4:junit4] 2> 4924 T2751 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1/index/
[junit4:junit4] 2> 4927 T2751 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 4928 T2751 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1/index forceNew:false
[junit4:junit4] 2> 4931 T2751 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@c85e682f lockFactory=org.apache.lucene.store.NativeFSLockFactory@b9f9f41b),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 4932 T2751 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 4933 T2751 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1/index
[junit4:junit4] 2> 4936 T2751 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 4937 T2751 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 4938 T2751 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 4940 T2751 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 4941 T2751 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 4942 T2751 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 4943 T2751 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 4945 T2751 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 4946 T2751 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 4947 T2751 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 4949 T2751 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1
[junit4:junit4] 2> 4950 T2751 oass.SolrIndexSearcher.<init> Opening Searcher@42c76c02 main
[junit4:junit4] 2> 4950 T2751 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1/tlog
[junit4:junit4] 2> 4951 T2751 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 4952 T2751 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 4955 T2752 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@42c76c02 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 4957 T2751 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 4957 T2751 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 6355 T2731 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 6356 T2731 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "base_url":"http://127.0.0.1:60021/jz/n",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "node_name":"127.0.0.1:60021_jz%2Fn",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 6357 T2731 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4] 2> 6357 T2731 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 6361 T2750 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 6361 T2737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 6361 T2730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 6958 T2751 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 6958 T2751 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:60021/jz/n collection:collection1 shard:shard1
[junit4:junit4] 2> 6959 T2751 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 6964 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70006 type:delete cxid:0x46 zxid:0x5b txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4] 2> 6964 T2751 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 6965 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70006 type:create cxid:0x47 zxid:0x5c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 6966 T2751 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 6966 T2751 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 6966 T2751 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:60021/jz/n/collection1/
[junit4:junit4] 2> 6967 T2751 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 6967 T2751 oasc.SyncStrategy.syncToMe http://127.0.0.1:60021/jz/n/collection1/ has no replicas
[junit4:junit4] 2> 6967 T2751 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:60021/jz/n/collection1/
[junit4:junit4] 2> 6968 T2751 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 6970 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70006 type:create cxid:0x51 zxid:0x60 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 7865 T2731 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 7896 T2737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 7896 T2730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 7896 T2750 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 7924 T2751 oasc.ZkController.register We are http://127.0.0.1:60021/jz/n/collection1/ and leader is http://127.0.0.1:60021/jz/n/collection1/
[junit4:junit4] 2> 7925 T2751 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:60021/jz/n
[junit4:junit4] 2> 7926 T2751 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 7926 T2751 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 7927 T2751 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 7929 T2751 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 7931 T2708 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 7932 T2708 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 7933 T2708 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 8005 T2708 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 8007 T2708 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49462
[junit4:junit4] 2> 8008 T2708 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 8008 T2708 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 8009 T2708 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362265684150
[junit4:junit4] 2> 8009 T2708 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362265684150/solr.xml
[junit4:junit4] 2> 8010 T2708 oasc.CoreContainer.<init> New CoreContainer 1479167609
[junit4:junit4] 2> 8010 T2708 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362265684150/'
[junit4:junit4] 2> 8011 T2708 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362265684150/'
[junit4:junit4] 2> 8022 T2708 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 8023 T2708 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 8023 T2708 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 8024 T2708 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 8024 T2708 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 8025 T2708 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 8026 T2708 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 8026 T2708 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 8027 T2708 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 8027 T2708 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 8034 T2708 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 8044 T2708 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:54711/solr
[junit4:junit4] 2> 8044 T2708 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 8045 T2708 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54711 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@b8a0312d
[junit4:junit4] 2> 8046 T2763 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54711. Will not attempt to authenticate using SASL (Access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 8046 T2708 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 8046 T2763 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54711, initiating session
[junit4:junit4] 2> 8046 T2710 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:54936
[junit4:junit4] 2> 8047 T2710 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:54936
[junit4:junit4] 2> 8048 T2712 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d2d5c99b70007 with negotiated timeout 20000 for client /127.0.0.1:54936
[junit4:junit4] 2> 8048 T2763 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54711, sessionid = 0x13d2d5c99b70007, negotiated timeout = 20000
[junit4:junit4] 2> 8048 T2764 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b8a0312d name:ZooKeeperConnection Watcher:127.0.0.1:54711 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 8049 T2708 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 8049 T2713 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d2d5c99b70007
[junit4:junit4] 2> 8050 T2710 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:54936 which had sessionid 0x13d2d5c99b70007
[junit4:junit4] 2> 8050 T2764 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 8050 T2708 oaz.ZooKeeper.close Session: 0x13d2d5c99b70007 closed
[junit4:junit4] 2> 8051 T2708 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 8055 T2708 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54711/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@3300a9fe
[junit4:junit4] 2> 8056 T2765 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54711. Will not attempt to authenticate using SASL (Access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 8056 T2708 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 8056 T2765 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54711, initiating session
[junit4:junit4] 2> 8056 T2710 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:54937
[junit4:junit4] 2> 8057 T2710 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:54937
[junit4:junit4] 2> 8058 T2712 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d2d5c99b70008 with negotiated timeout 20000 for client /127.0.0.1:54937
[junit4:junit4] 2> 8058 T2765 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54711, sessionid = 0x13d2d5c99b70008, negotiated timeout = 20000
[junit4:junit4] 2> 8058 T2766 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3300a9fe name:ZooKeeperConnection Watcher:127.0.0.1:54711/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 8059 T2708 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 8059 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70008 type:create cxid:0x1 zxid:0x6d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 8060 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70008 type:create cxid:0x2 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 8061 T2708 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 9063 T2708 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49462_jz%2Fn
[junit4:junit4] 2> 9064 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70008 type:delete cxid:0x9 zxid:0x6f txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:49462_jz%2Fn Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:49462_jz%2Fn
[junit4:junit4] 2> 9065 T2708 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49462_jz%2Fn
[junit4:junit4] 2> 9067 T2737 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9068 T2750 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9068 T2766 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9068 T2730 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9068 T2737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9068 T2750 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9069 T2730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9074 T2767 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362265684150/collection1
[junit4:junit4] 2> 9075 T2767 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 9075 T2767 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 9076 T2767 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 9077 T2767 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362265684150/collection1/'
[junit4:junit4] 2> 9078 T2767 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362265684150/collection1/lib/README' to classloader
[junit4:junit4] 2> 9079 T2767 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362265684150/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 9107 T2767 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 9139 T2767 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 9140 T2767 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 9145 T2767 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 9403 T2731 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 9404 T2731 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "base_url":"http://127.0.0.1:60021/jz/n",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "node_name":"127.0.0.1:60021_jz%2Fn",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 9406 T2737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9406 T2766 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9406 T2750 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9406 T2730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9483 T2767 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 9488 T2767 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 9491 T2767 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 9505 T2767 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 9510 T2767 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 9513 T2767 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 9514 T2767 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 9514 T2767 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 9514 T2767 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 9515 T2767 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 9516 T2767 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 9516 T2767 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1362265684150/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty2/
[junit4:junit4] 2> 9516 T2767 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3de91d1e
[junit4:junit4] 2> 9517 T2767 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 9517 T2767 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty2 forceNew:false
[junit4:junit4] 2> 9518 T2767 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty2
[junit4:junit4] 2> 9518 T2767 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty2/index/
[junit4:junit4] 2> 9519 T2767 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 9519 T2767 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty2/index forceNew:false
[junit4:junit4] 2> 9520 T2767 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@e8168904 lockFactory=org.apache.lucene.store.NativeFSLockFactory@87db2c99),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 9521 T2767 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 9521 T2767 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty2/index
[junit4:junit4] 2> 9522 T2767 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 9523 T2767 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 9523 T2767 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 9524 T2767 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 9524 T2767 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 9524 T2767 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 9525 T2767 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 9525 T2767 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 9526 T2767 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 9527 T2767 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 9529 T2767 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty2
[junit4:junit4] 2> 9529 T2767 oass.SolrIndexSearcher.<init> Opening Searcher@32ec4ed4 main
[junit4:junit4] 2> 9530 T2767 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty2/tlog
[junit4:junit4] 2> 9530 T2767 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 9531 T2767 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 9535 T2768 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@32ec4ed4 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 9537 T2767 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 9537 T2767 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 10912 T2731 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 10913 T2731 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49462/jz/n",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "node_name":"127.0.0.1:49462_jz%2Fn",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 10913 T2731 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 10914 T2731 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4] 2> 10917 T2737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 10917 T2766 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 10917 T2750 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 10918 T2730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 11539 T2767 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 11539 T2767 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:49462/jz/n collection:collection1 shard:shard2
[junit4:junit4] 2> 11540 T2767 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 11544 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70008 type:delete cxid:0x45 zxid:0x7e txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
[junit4:junit4] 2> 11545 T2767 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 11545 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70008 type:create cxid:0x46 zxid:0x7f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 11547 T2767 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 11548 T2767 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 11548 T2767 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:49462/jz/n/collection1/
[junit4:junit4] 2> 11549 T2767 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 11549 T2767 oasc.SyncStrategy.syncToMe http://127.0.0.1:49462/jz/n/collection1/ has no replicas
[junit4:junit4] 2> 11550 T2767 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:49462/jz/n/collection1/
[junit4:junit4] 2> 11550 T2767 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 11553 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70008 type:create cxid:0x4f zxid:0x82 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 12421 T2731 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 12426 T2766 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 12426 T2737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 12426 T2750 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 12426 T2730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 12456 T2767 oasc.ZkController.register We are http://127.0.0.1:49462/jz/n/collection1/ and leader is http://127.0.0.1:49462/jz/n/collection1/
[junit4:junit4] 2> 12457 T2767 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49462/jz/n
[junit4:junit4] 2> 12458 T2767 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 12458 T2767 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 12459 T2767 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 12461 T2767 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 12462 T2708 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 12464 T2708 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 12465 T2708 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 12541 T2708 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 12543 T2708 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:32892
[junit4:junit4] 2> 12544 T2708 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 12544 T2708 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 12545 T2708 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362265688675
[junit4:junit4] 2> 12545 T2708 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362265688675/solr.xml
[junit4:junit4] 2> 12546 T2708 oasc.CoreContainer.<init> New CoreContainer 1878696378
[junit4:junit4] 2> 12547 T2708 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362265688675/'
[junit4:junit4] 2> 12547 T2708 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362265688675/'
[junit4:junit4] 2> 12561 T2708 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 12562 T2708 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 12563 T2708 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 12563 T2708 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 12564 T2708 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 12564 T2708 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 12565 T2708 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 12566 T2708 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 12566 T2708 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 12567 T2708 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 12576 T2708 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 12592 T2708 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:54711/solr
[junit4:junit4] 2> 12593 T2708 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 12594 T2708 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54711 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@ebb8185d
[junit4:junit4] 2> 12595 T2779 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54711. Will not attempt to authenticate using SASL (Access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 12595 T2708 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 12595 T2779 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54711, initiating session
[junit4:junit4] 2> 12595 T2710 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:54942
[junit4:junit4] 2> 12596 T2710 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:54942
[junit4:junit4] 2> 12597 T2712 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d2d5c99b70009 with negotiated timeout 20000 for client /127.0.0.1:54942
[junit4:junit4] 2> 12597 T2779 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54711, sessionid = 0x13d2d5c99b70009, negotiated timeout = 20000
[junit4:junit4] 2> 12598 T2780 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ebb8185d name:ZooKeeperConnection Watcher:127.0.0.1:54711 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 12598 T2708 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 12599 T2713 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d2d5c99b70009
[junit4:junit4] 2> 12600 T2710 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:54942 which had sessionid 0x13d2d5c99b70009
[junit4:junit4] 2> 12600 T2708 oaz.ZooKeeper.close Session: 0x13d2d5c99b70009 closed
[junit4:junit4] 2> 12600 T2780 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 12601 T2708 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 12605 T2708 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54711/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@9da9e782
[junit4:junit4] 2> 12606 T2781 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54711. Will not attempt to authenticate using SASL (Access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 12606 T2708 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 12607 T2710 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:54943
[junit4:junit4] 2> 12607 T2781 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54711, initiating session
[junit4:junit4] 2> 12608 T2710 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:54943
[junit4:junit4] 2> 12609 T2712 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d2d5c99b7000a with negotiated timeout 20000 for client /127.0.0.1:54943
[junit4:junit4] 2> 12609 T2781 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54711, sessionid = 0x13d2d5c99b7000a, negotiated timeout = 20000
[junit4:junit4] 2> 12609 T2782 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@9da9e782 name:ZooKeeperConnection Watcher:127.0.0.1:54711/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 12609 T2708 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 12610 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b7000a type:create cxid:0x1 zxid:0x8f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 12611 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b7000a type:create cxid:0x2 zxid:0x90 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 12612 T2708 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 13614 T2708 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:32892_jz%2Fn
[junit4:junit4] 2> 13617 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b7000a type:delete cxid:0x9 zxid:0x91 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:32892_jz%2Fn Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:32892_jz%2Fn
[junit4:junit4] 2> 13618 T2708 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:32892_jz%2Fn
[junit4:junit4] 2> 13623 T2766 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 13623 T2730 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 13623 T2750 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 13623 T2782 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 13624 T2766 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 13624 T2750 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 13624 T2737 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 13626 T2737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 13626 T2730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 13634 T2783 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362265688675/collection1
[junit4:junit4] 2> 13635 T2783 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 13636 T2783 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 13637 T2783 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 13639 T2783 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362265688675/collection1/'
[junit4:junit4] 2> 13640 T2783 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362265688675/collection1/lib/README' to classloader
[junit4:junit4] 2> 13641 T2783 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362265688675/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 13680 T2783 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 13712 T2783 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 13713 T2783 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 13718 T2783 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 13931 T2731 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 13931 T2731 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49462/jz/n",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "node_name":"127.0.0.1:49462_jz%2Fn",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 13935 T2730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 13935 T2782 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 13935 T2750 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 13935 T2766 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 13937 T2737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14074 T2783 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 14079 T2783 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 14082 T2783 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 14093 T2783 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 14096 T2783 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 14101 T2783 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 14102 T2783 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 14103 T2783 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 14103 T2783 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 14105 T2783 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 14105 T2783 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 14106 T2783 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1362265688675/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3/
[junit4:junit4] 2> 14106 T2783 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3de91d1e
[junit4:junit4] 2> 14107 T2783 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 14108 T2783 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3 forceNew:false
[junit4:junit4] 2> 14109 T2783 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3
[junit4:junit4] 2> 14109 T2783 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3/index/
[junit4:junit4] 2> 14110 T2783 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 14111 T2783 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3/index forceNew:false
[junit4:junit4] 2> 14113 T2783 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@17091b05 lockFactory=org.apache.lucene.store.NativeFSLockFactory@58c80795),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 14113 T2783 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 14114 T2783 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3/index
[junit4:junit4] 2> 14116 T2783 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 14116 T2783 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 14117 T2783 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 14118 T2783 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 14119 T2783 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 14120 T2783 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 14120 T2783 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 14121 T2783 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 14122 T2783 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 14123 T2783 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 14126 T2783 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3
[junit4:junit4] 2> 14127 T2783 oass.SolrIndexSearcher.<init> Opening Searcher@ae3b5894 main
[junit4:junit4] 2> 14127 T2783 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3/tlog
[junit4:junit4] 2> 14128 T2783 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 14129 T2783 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 14135 T2784 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@ae3b5894 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 14138 T2783 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 14138 T2783 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 15440 T2731 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 15441 T2731 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "base_url":"http://127.0.0.1:32892/jz/n",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "node_name":"127.0.0.1:32892_jz%2Fn",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 15441 T2731 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 15441 T2731 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 15446 T2737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 15446 T2750 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 15446 T2782 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 15446 T2730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 15446 T2766 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 16139 T2783 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 16140 T2783 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:32892/jz/n collection:collection1 shard:shard1
[junit4:junit4] 2> 16143 T2783 oasc.ZkController.register We are http://127.0.0.1:32892/jz/n/collection1/ and leader is http://127.0.0.1:60021/jz/n/collection1/
[junit4:junit4] 2> 16143 T2783 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:32892/jz/n
[junit4:junit4] 2> 16144 T2783 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 16144 T2783 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C176 name=collection1 org.apache.solr.core.SolrCore@18e6b08e url=http://127.0.0.1:32892/jz/n/collection1 node=127.0.0.1:32892_jz%2Fn C176_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:32892/jz/n, state=down, node_name=127.0.0.1:32892_jz%2Fn, collection=collection1, core=collection1}
[junit4:junit4] 2> 16145 T2785 C176 P32892 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 16145 T2785 C176 P32892 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 16145 T2783 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 16145 T2785 C176 P32892 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 16146 T2785 C176 P32892 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 16146 T2708 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 16147 T2708 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 16148 T2708 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 16148 T2785 C176 P32892 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 16154 T2708 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 16155 T2708 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 16163 T2746 oasc.CoreContainer.create Creating SolrCore 'onenodecollectioncore' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362265679580/onenodecollectioncore
[junit4:junit4] 2> 16164 T2746 oasc.ZkController.createCollectionZkNode Check for collection zkNode:onenodecollection
[junit4:junit4] 2> 16165 T2746 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:onenodecollection
[junit4:junit4] 2> 16166 T2746 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 16167 T2746 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 16168 T2746 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection
[junit4:junit4] 2> 16170 T2746 oasc.ZkController.readConfigName Load collection config from:/collections/onenodecollection
[junit4:junit4] 2> 16172 T2746 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362265679580/onenodecollectioncore/'
[junit4:junit4] 2> 16208 T2746 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 16254 T2746 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 16255 T2746 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 16262 T2746 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 16656 T2746 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 16662 T2746 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 16666 T2746 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 16677 T2746 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 16682 T2746 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 16686 T2746 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 16688 T2746 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 16688 T2746 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 16689 T2746 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 16690 T2746 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 16691 T2746 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 16692 T2746 oasc.SolrCore.<init> [onenodecollectioncore] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1362265679580/onenodecollectioncore/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362265676210/onenodecollection/
[junit4:junit4] 2> 16693 T2746 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3de91d1e
[junit4:junit4] 2> 16694 T2746 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 16695 T2746 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362265676210/onenodecollection forceNew:false
[junit4:junit4] 2> 16696 T2746 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362265676210/onenodecollection
[junit4:junit4] 2> 16696 T2746 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362265676210/onenodecollection/index/
[junit4:junit4] 2> 16697 T2746 oasc.SolrCore.initIndex WARNING [onenodecollectioncore] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362265676210/onenodecollection/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 16698 T2746 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362265676210/onenodecollection/index forceNew:false
[junit4:junit4] 2> 16700 T2746 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@18b16779 lockFactory=org.apache.lucene.store.NativeFSLockFactory@e97411df),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 16700 T2746 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 16701 T2746 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362265676210/onenodecollection/index
[junit4:junit4] 2> 16703 T2746 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 16704 T2746 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 16705 T2746 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 16706 T2746 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 16707 T2746 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 16707 T2746 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 16708 T2746 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 16709 T2746 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 16710 T2746 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 16711 T2746 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 16713 T2746 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362265676210/onenodecollection
[junit4:junit4] 2> 16714 T2746 oass.SolrIndexSearcher.<init> Opening Searcher@7559b3d3 main
[junit4:junit4] 2> 16715 T2746 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362265676210/onenodecollection/tlog
[junit4:junit4] 2> 16716 T2746 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 16717 T2746 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 16722 T2787 oasc.SolrCore.registerSearcher [onenodecollectioncore] Registered new searcher Searcher@7559b3d3 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 16725 T2746 oasc.ZkController.publish publishing core=onenodecollectioncore state=down
[junit4:junit4] 2> 16950 T2731 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 16951 T2731 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "base_url":"http://127.0.0.1:32892/jz/n",
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "node_name":"127.0.0.1:32892_jz%2Fn",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 16955 T2731 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "base_url":"http://127.0.0.1:60021/jz/n",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:60021_jz%2Fn",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"onenodecollectioncore"}
[junit4:junit4] 2> 16956 T2731 oasc.Overseer$ClusterStateUpdater.createCollection Create collection onenodecollection with numShards 1
[junit4:junit4] 2> 16956 T2731 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 16961 T2737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 16961 T2750 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 16962 T2766 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 16962 T2730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 16963 T2782 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17727 T2746 oasc.CoreContainer.registerCore registering core: onenodecollectioncore
[junit4:junit4] 2> 17728 T2746 oasc.ZkController.register Register replica - core:onenodecollectioncore address:http://127.0.0.1:60021/jz/n collection:onenodecollection shard:shard1
[junit4:junit4] 2> 17729 T2746 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection/leader_elect/shard1/election
[junit4:junit4] 2> 17735 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70006 type:delete cxid:0x9f zxid:0xac txntype:-1 reqpath:n/a Error Path:/solr/collections/onenodecollection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/onenodecollection/leaders
[junit4:junit4] 2> 17737 T2746 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 17737 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70006 type:create cxid:0xa0 zxid:0xad txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 17740 T2746 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 17741 T2746 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 17742 T2746 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:60021/jz/n/onenodecollectioncore/
[junit4:junit4] 2> 17742 T2746 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 17743 T2746 oasc.SyncStrategy.syncToMe http://127.0.0.1:60021/jz/n/onenodecollectioncore/ has no replicas
[junit4:junit4] 2> 17744 T2746 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:60021/jz/n/onenodecollectioncore/
[junit4:junit4] 2> 17744 T2746 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection/leaders/shard1
[junit4:junit4] 2> 17748 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70006 type:create cxid:0xaa zxid:0xb1 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 18469 T2731 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 18474 T2750 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 18474 T2737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 18474 T2766 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 18474 T2730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 18474 T2782 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 18500 T2746 oasc.ZkController.register We are http://127.0.0.1:60021/jz/n/onenodecollectioncore/ and leader is http://127.0.0.1:60021/jz/n/onenodecollectioncore/
[junit4:junit4] 2> 18501 T2746 oasc.ZkController.register No LogReplay needed for core=onenodecollectioncore baseURL=http://127.0.0.1:60021/jz/n
[junit4:junit4] 2> 18502 T2746 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 18503 T2746 oasc.ZkController.publish publishing core=onenodecollectioncore state=active
[junit4:junit4] 2> 18504 T2746 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 18505 T2708 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: onenodecollection failOnTimeout:true timeout (sec):230
[junit4:junit4] 2> 18506 T2708 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C176_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:32892/jz/n, state=recovering, node_name=127.0.0.1:32892_jz%2Fn, collection=collection1, shard=shard1, core=collection1}
[junit4:junit4] 2> 19156 T2785 C176 P32892 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:60021/jz/n/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 19157 T2785 C176 P32892 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:32892/jz/n START replicas=[http://127.0.0.1:60021/jz/n/collection1/] nUpdates=100
[junit4:junit4] 2> 19157 T2785 C176 P32892 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 19158 T2785 C176 P32892 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 19158 T2785 C176 P32892 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 19158 T2785 C176 P32892 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 19159 T2785 C176 P32892 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 19159 T2785 C176 P32892 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:60021/jz/n/collection1/. core=collection1
[junit4:junit4] 2> 19159 T2785 C176 P32892 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C177 name=collection1 org.apache.solr.core.SolrCore@5465695e url=http://127.0.0.1:60021/jz/n/collection1 node=127.0.0.1:60021_jz%2Fn C177_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:60021/jz/n, state=active, node_name=127.0.0.1:60021_jz%2Fn, collection=collection1, shard=shard1, core=collection1, leader=true}
[junit4:junit4] 2> 19353 T2744 C177 P60021 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 19354 T2743 C177 P60021 REQ /get {getVersions=100&qt=/get&distrib=false&version=2&wt=javabin} status=0 QTime=0
[junit4:junit4] 2> 19354 T2744 C177 P60021 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1
[junit4:junit4] 2> 19356 T2744 C177 P60021 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@c85e682f lockFactory=org.apache.lucene.store.NativeFSLockFactory@b9f9f41b),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 19357 T2744 C177 P60021 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 19358 T2744 C177 P60021 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@c85e682f lockFactory=org.apache.lucene.store.NativeFSLockFactory@b9f9f41b),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@c85e682f lockFactory=org.apache.lucene.store.NativeFSLockFactory@b9f9f41b),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> 19358 T2744 C177 P60021 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4] 2> 19359 T2744 C177 P60021 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1
[junit4:junit4] 2> 19372 T2744 C177 P60021 oass.SolrIndexSearcher.<init> Opening Searcher@25753d95 realtime
[junit4:junit4] 2> 19373 T2744 C177 P60021 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 19375 T2744 C177 P60021 /update {softCommit=false&commit_end_point=true&commit=true&version=2&waitSearcher=true&wt=javabin&openSearcher=false} {commit=} 0 22
[junit4:junit4] 2> 19376 T2785 C176 P32892 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 19376 T2785 C176 P32892 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 19378 T2743 C177 P60021 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 19379 T2743 C177 P60021 REQ /replication {qt=/replication&version=2&command=indexversion&wt=javabin} status=0 QTime=2
[junit4:junit4] 2> 19380 T2785 C176 P32892 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 19380 T2785 C176 P32892 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 19380 T2785 C176 P32892 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 19382 T2746 C177 P60021 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1
[junit4:junit4] 2> 19383 T2746 C177 P60021 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1/index
[junit4:junit4] 2> 19384 T2746 C177 P60021 REQ /replication {qt=/replication&version=2&generation=2&command=filelist&wt=javabin} status=0 QTime=2
[junit4:junit4] 2> 19385 T2785 C176 P32892 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 19386 T2785 C176 P32892 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3/index.20130302200815591 forceNew:false
[junit4:junit4] 2> 19387 T2785 C176 P32892 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3
[junit4:junit4] 2> 19387 T2785 C176 P32892 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@48b5c8c0 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4a1bdf41) fullCopy=false
[junit4:junit4] 2> 19389 T2744 C177 P60021 REQ /replication {checksum=true&qt=/replication&generation=2&command=filecontent&file=segments_2&wt=filestream} status=0 QTime=0
[junit4:junit4] 2> 19390 T2785 C176 P32892 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3/index
[junit4:junit4] 2> 19391 T2785 C176 P32892 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 19391 T2785 C176 P32892 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3
[junit4:junit4] 2> 19392 T2785 C176 P32892 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3
[junit4:junit4] 2> 19392 T2785 C176 P32892 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 19392 T2785 C176 P32892 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 19392 T2785 C176 P32892 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3
[junit4:junit4] 2> 19393 T2785 C176 P32892 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@17091b05 lockFactory=org.apache.lucene.store.NativeFSLockFactory@58c80795),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@17091b05 lockFactory=org.apache.lucene.store.NativeFSLockFactory@58c80795),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> 19394 T2785 C176 P32892 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4] 2> 19394 T2785 C176 P32892 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 19394 T2785 C176 P32892 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3
[junit4:junit4] 2> 19395 T2785 C176 P32892 oass.SolrIndexSearcher.<init> Opening Searcher@b5797695 main
[junit4:junit4] 2> 19396 T2784 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@b5797695 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 19396 T2784 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3/index
[junit4:junit4] 2> 19396 T2785 C176 P32892 oash.SnapPuller.fetchLatestIndex removing temporary index download directory files MockDirWrapper(org.apache.lucene.store.RAMDirectory@48b5c8c0 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4a1bdf41)
[junit4:junit4] 2> 19397 T2785 C176 P32892 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3/index.20130302200815591
[junit4:junit4] 2> 19397 T2785 C176 P32892 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3/index
[junit4:junit4] 2> 19397 T2785 C176 P32892 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 19397 T2785 C176 P32892 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 19398 T2785 C176 P32892 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 19398 T2785 C176 P32892 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 19399 T2785 C176 P32892 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 19508 T2708 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 19979 T2731 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 19980 T2731 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "base_url":"http://127.0.0.1:60021/jz/n",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:60021_jz%2Fn",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"onenodecollectioncore"}
[junit4:junit4] 2> 19983 T2731 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "base_url":"http://127.0.0.1:32892/jz/n",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "node_name":"127.0.0.1:32892_jz%2Fn",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 19986 T2737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 19986 T2782 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 19986 T2750 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 19986 T2730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 19986 T2766 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 20510 T2708 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 20511 T2708 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: onenodecollection
[junit4:junit4] 2> 20512 T2708 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> C176_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:32892/jz/n, state=active, node_name=127.0.0.1:32892_jz%2Fn, collection=collection1, shard=shard1, core=collection1}
[junit4:junit4] 2> 20547 T2772 C176 P32892 /update {update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:60021/jz/n/collection1/&version=2&wt=javabin} {add=[1 (1428439115231133696)]} 0 1
[junit4:junit4] 2> 20548 T2745 C177 P60021 /update {version=2&wt=javabin} {add=[1 (1428439115231133696)]} 0 7
[junit4:junit4] 2> 20669 T2746 C177 P60021 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 20671 T2746 C177 P60021 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@c85e682f lockFactory=org.apache.lucene.store.NativeFSLockFactory@b9f9f41b),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@c85e682f lockFactory=org.apache.lucene.store.NativeFSLockFactory@b9f9f41b),segFN=segments_3,generation=3,filenames=[_0_MockVariableIntBlock_0.tib, _0_Asserting_0.dvd, _0.fnm, _0.nvm, _0_MockVariableIntBlock_0.pos, _0_MockVariableIntBlock_0.tii, _0_Asserting_0.dvm, _0_MockVariableIntBlock_0.skp, _0_MockVariableIntBlock_0.doc, _0_Direct_0.pos, _0_Memory_0.ram, _0_Memory_1.ram, _0_Direct_0.tim, _0_Direct_0.doc, _0.si, segments_3, _0_Direct_0.tip, _0_MockVariableIntBlock_0.pyl, _0.fdt, _0_MockVariableIntBlock_0.frq, _0.fdx, _0.nvd]
[junit4:junit4] 2> 20672 T2746 C177 P60021 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[_0_MockVariableIntBlock_0.tib, _0_Asserting_0.dvd, _0.fnm, _0.nvm, _0_MockVariableIntBlock_0.pos, _0_MockVariableIntBlock_0.tii, _0_Asserting_0.dvm, _0_MockVariableIntBlock_0.skp, _0_MockVariableIntBlock_0.doc, _0_Direct_0.pos, _0_Memory_0.ram, _0_Memory_1.ram, _0_Direct_0.tim, _0_Direct_0.doc, _0.si, segments_3, _0_Direct_0.tip, _0_MockVariableIntBlock_0.pyl, _0.fdt, _0_MockVariableIntBlock_0.frq, _0.fdx, _0.nvd]
[junit4:junit4] 2> 20673 T2746 C177 P60021 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1
[junit4:junit4] 2> 20674 T2746 C177 P60021 oass.SolrIndexSearcher.<init> Opening Searcher@ffd40ae5 main
[junit4:junit4] 2> 20675 T2746 C177 P60021 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1/index
[junit4:junit4] 2> 20676 T2746 C177 P60021 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 20677 T2752 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@ffd40ae5 main{StandardDirectoryReader(segments_3:3:nrt _0(5.0):C1)}
[junit4:junit4] 2> 20678 T2752 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1/index
[junit4:junit4] 2> 20678 T2746 C177 P60021 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:32892/jz/n/collection1/, StdNode: http://127.0.0.1:49462/jz/n/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 20680 T2773 C176 P32892 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 20682 T2773 C176 P32892 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@17091b05 lockFactory=org.apache.lucene.store.NativeFSLockFactory@58c80795),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@17091b05 lockFactory=org.apache.lucene.store.NativeFSLockFactory@58c80795),segFN=segments_3,generation=3,filenames=[_0_MockVariableIntBlock_0.tib, _0_Asserting_0.dvd, _0.fnm, _0.nvm, _0_MockVariableIntBlock_0.pos, _0_MockVariableIntBlock_0.tii, _0_Asserting_0.dvm, _0_MockVariableIntBlock_0.skp, _0_MockVariableIntBlock_0.doc, _0_Direct_0.pos, _0_Memory_0.ram, _0_Memory_1.ram, _0_Direct_0.tim, _0_Direct_0.doc, _0.si, segments_3, _0_Direct_0.tip, _0_MockVariableIntBlock_0.pyl, _0.fdt, _0_MockVariableIntBlock_0.frq, _0.fdx, _0.nvd]
[junit4:junit4] 2> 20683 T2773 C176 P32892 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[_0_MockVariableIntBlock_0.tib, _0_Asserting_0.dvd, _0.fnm, _0.nvm, _0_MockVariableIntBlock_0.pos, _0_MockVariableIntBlock_0.tii, _0_Asserting_0.dvm, _0_MockVariableIntBlock_0.skp, _0_MockVariableIntBlock_0.doc, _0_Direct_0.pos, _0_Memory_0.ram, _0_Memory_1.ram, _0_Direct_0.tim, _0_Direct_0.doc, _0.si, segments_3, _0_Direct_0.tip, _0_MockVariableIntBlock_0.pyl, _0.fdt, _0_MockVariableIntBlock_0.frq, _0.fdx, _0.nvd]
[junit4:junit4] 2> ASYNC NEW_CORE C178 name=collection1 org.apache.solr.core.SolrCore@cf8d8b14 url=http://127.0.0.1:49462/jz/n/collection1 node=127.0.0.1:49462_jz%2Fn C178_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:49462/jz/n, state=active, node_name=127.0.0.1:49462_jz%2Fn, collection=collection1, shard=shard2, core=collection1, leader=true}
[junit4:junit4] 2> 20683 T2759 C178 P49462 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 20684 T2773 C176 P32892 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3
[junit4:junit4] 2> 20685 T2759 C178 P49462 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty2
[junit4:junit4] 2> 20686 T2773 C176 P32892 oass.SolrIndexSearcher.<init> Opening Searcher@c6fdba14 main
[junit4:junit4] 2> 20687 T2759 C178 P49462 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@e8168904 lockFactory=org.apache.lucene.store.NativeFSLockFactory@87db2c99),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 20687 T2773 C176 P32892 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 20687 T2784 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@c6fdba14 main{StandardDirectoryReader(segments_3:3:nrt _0(5.0):C1)}
[junit4:junit4] 2> 20687 T2759 C178 P49462 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 20689 T2784 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3/index
[junit4:junit4] 2> 20689 T2759 C178 P49462 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@e8168904 lockFactory=org.apache.lucene.store.NativeFSLockFactory@87db2c99),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@e8168904 lockFactory=org.apache.lucene.store.NativeFSLockFactory@87db2c99),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> 20689 T2773 C176 P32892 /update {softCommit=false&expungeDeletes=false&commit_end_point=true&commit=true&version=2&waitSearcher=true&wt=javabin} {commit=} 0 9
[junit4:junit4] 2> 20690 T2759 C178 P49462 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4] 2> 20691 T2759 C178 P49462 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty2
[junit4:junit4] 2> 20692 T2759 C178 P49462 oass.SolrIndexSearcher.<init> Opening Searcher@cced56cf main
[junit4:junit4] 2> 20693 T2759 C178 P49462 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 20693 T2768 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@cced56cf main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 20695 T2768 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty2/index
[junit4:junit4] 2> 20695 T2759 C178 P49462 /update {softCommit=false&expungeDeletes=false&commit_end_point=true&commit=true&version=2&waitSearcher=true&wt=javabin} {commit=} 0 12
[junit4:junit4] 2> 20696 T2746 C177 P60021 /update {softCommit=false&commit=true&version=2&waitSearcher=true&wt=javabin} {commit=} 0 27
[junit4:junit4] 2> 20702 T2759 C178 P49462 REQ /select {q=*:*&start=0&isShard=true&distrib=false&NOW=1362265696904&fl=id,score&version=2&fsv=true&rows=10&wt=javabin&shard.url=127.0.0.1:49462/jz/n/collection1/} hits=0 status=0 QTime=0
[junit4:junit4] 2> 20702 T2776 C176 P32892 REQ /select {q=*:*&start=0&isShard=true&distrib=false&NOW=1362265696904&fl=id,score&version=2&fsv=true&rows=10&wt=javabin&shard.url=127.0.0.1:60021/jz/n/collection1/|127.0.0.1:32892/jz/n/collection1/} hits=1 status=0 QTime=0
[junit4:junit4] 2> 20705 T2775 C176 P32892 REQ /select {q=*:*&isShard=true&distrib=false&NOW=1362265696904&version=2&ids=1&wt=javabin&shard.url=127.0.0.1:60021/jz/n/collection1/|127.0.0.1:32892/jz/n/collection1/} status=0 QTime=0
[junit4:junit4] 2> 20706 T2745 C177 P60021 REQ /select {q=*:*&version=2&wt=javabin} hits=1 status=0 QTime=7
[junit4:junit4] 2> 20708 T2708 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C179 name=onenodecollectioncore org.apache.solr.core.SolrCore@ffa0d4fa url=http://127.0.0.1:60021/jz/n/onenodecollectioncore node=127.0.0.1:60021_jz%2Fn C179_STATE=coll:onenodecollection core:onenodecollectioncore props:{base_url=http://127.0.0.1:60021/jz/n, state=active, node_name=127.0.0.1:60021_jz%2Fn, roles=none, collection=onenodecollection, shard=shard1, core=onenodecollectioncore, leader=true}
[junit4:junit4] 2> 20717 T2743 C179 P60021 REQ /select {q=*:*&version=2&wt=javabin} hits=0 status=0 QTime=0
[junit4:junit4] 2> 20719 T2708 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54711 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@bafd331c
[junit4:junit4] 2> 20720 T2708 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 20720 T2795 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54711. Will not attempt to authenticate using SASL (Access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 20721 T2710 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:54958
[junit4:junit4] 2> 21193 T2795 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54711, initiating session
[junit4:junit4] 2> 21497 T2710 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:54958
[junit4:junit4] 2> 21498 T2712 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d2d5c99b7000b with negotiated timeout 10000 for client /127.0.0.1:54958
[junit4:junit4] 2> 21498 T2795 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54711, sessionid = 0x13d2d5c99b7000b, negotiated timeout = 10000
[junit4:junit4] 2> 21498 T2796 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@bafd331c name:ZooKeeperConnection Watcher:127.0.0.1:54711 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 21499 T2708 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 21558 T2713 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d2d5c99b7000b
[junit4:junit4] 2> 21560 T2710 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:54958 which had sessionid 0x13d2d5c99b7000b
[junit4:junit4] 2> 21560 T2796 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 21560 T2708 oaz.ZooKeeper.close Session: 0x13d2d5c99b7000b closed
[junit4:junit4] 2> 21561 T2713 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d2d5c99b70004
[junit4:junit4] 2> 21562 T2710 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:54928 which had sessionid 0x13d2d5c99b70004
[junit4:junit4] 2> 21562 T2737 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 21562 T2708 oaz.ZooKeeper.close Session: 0x13d2d5c99b70004 closed
[junit4:junit4] 2> 21566 T2708 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 34136
[junit4:junit4] 2> 21566 T2708 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1112915197
[junit4:junit4] 2> 23000 T2731 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 23001 T2731 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34136/jz/n",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "node_name":"127.0.0.1:34136_jz%2Fn",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 23004 T2730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 23005 T2782 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 23004 T2750 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 23004 T2766 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 24568 T2708 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 24569 T2708 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 24572 T2708 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@d99a2806
[junit4:junit4] 2> 24575 T2708 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3/index
[junit4:junit4] 2> 24577 T2708 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3
[junit4:junit4] 2> 24579 T2708 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 24580 T2708 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 24580 T2708 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 24581 T2708 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 24582 T2708 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 24583 T2708 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/control/data/index
[junit4:junit4] 2> 24584 T2708 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/control/data
[junit4:junit4] 2> 24585 T2708 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/control/data
[junit4:junit4] 2> 24586 T2708 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/control/data/index
[junit4:junit4] 2> 24586 T2708 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/control/data/index
[junit4:junit4] 2> 24588 T2731 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89277443356426243-127.0.0.1:34136_jz%2Fn-n_0000000000) am no longer a leader.
[junit4:junit4] 2> 24588 T2713 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d2d5c99b70003
[junit4:junit4] 2> 24589 T2710 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:54923 which had sessionid 0x13d2d5c99b70003
[junit4:junit4] 2> 24590 T2708 oaz.ZooKeeper.close Session: 0x13d2d5c99b70003 closed
[junit4:junit4] 2> 24590 T2766 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 24590 T2782 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 24591 T2766 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 24591 T2782 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 24590 T2750 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 24592 T2750 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 24595 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70006 type:delete cxid:0xba zxid:0xcd txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4] 2> 24595 T2750 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 24597 T2750 oasc.Overseer.start Overseer (id=89277443356426246-127.0.0.1:60021_jz%2Fn-n_0000000001) starting
[junit4:junit4] 2> 24598 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70006 type:create cxid:0xbf zxid:0xcf txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 24598 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70006 type:create cxid:0xc0 zxid:0xd0 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 24599 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70006 type:create cxid:0xc1 zxid:0xd1 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 24600 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70006 type:create cxid:0xc2 zxid:0xd2 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 24602 T2798 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 24604 T2797 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 24607 T2797 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 24607 T2797 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34136/jz/n",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "node_name":"127.0.0.1:34136_jz%2Fn",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 24613 T2766 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 24613 T2782 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 24614 T2750 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 24614 T2708 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/jz/n,null}
[junit4:junit4] 2> 24667 T2708 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 60021
[junit4:junit4] 2> 24667 T2708 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=-1556019191
[junit4:junit4] 2> 25890 T2730 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 25890 T2730 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 25890 T2730 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 25891 T2730 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 26117 T2797 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 26118 T2797 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "base_url":"http://127.0.0.1:60021/jz/n",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:60021_jz%2Fn_onenodecollectioncore",
[junit4:junit4] 2> "node_name":"127.0.0.1:60021_jz%2Fn",
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"onenodecollectioncore"}
[junit4:junit4] 2> 26121 T2797 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "base_url":"http://127.0.0.1:60021/jz/n",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "node_name":"127.0.0.1:60021_jz%2Fn",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 26125 T2766 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 26125 T2782 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 26125 T2750 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 27673 T2708 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 27673 T2708 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 27675 T2708 oasc.ZkController.publish publishing core=onenodecollectioncore state=down
[junit4:junit4] 2> 27678 T2708 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@5465695e
[junit4:junit4] 2> 27679 T2708 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3/index
[junit4:junit4] 2> 27680 T2708 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3
[junit4:junit4] 2> 27682 T2708 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=2,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=1,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 27683 T2708 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 27683 T2708 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 27684 T2708 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> C177_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:60021/jz/n, state=down, node_name=127.0.0.1:60021_jz%2Fn, collection=collection1, shard=shard1, core=collection1, leader=true}
[junit4:junit4] 2> 27685 T2708 C177 P60021 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1/index
[junit4:junit4] 2> 27685 T2708 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 27686 T2708 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1/index
[junit4:junit4] 2> 27687 T2708 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1/index
[junit4:junit4] 2> 27687 T2708 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1/index
[junit4:junit4] 2> 27688 T2708 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1
[junit4:junit4] 2> 27688 T2708 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty1
[junit4:junit4] 2> 27689 T2708 oasc.SolrCore.close [onenodecollectioncore] CLOSING SolrCore org.apache.solr.core.SolrCore@ffa0d4fa
[junit4:junit4] 2> 27694 T2708 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 27695 T2708 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 27696 T2708 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 27697 T2708 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 27697 T2708 oasc.SolrCore.closeSearcher [onenodecollectioncore] Closing main searcher on request.
[junit4:junit4] 2> 27698 T2708 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362265676210/onenodecollection/index
[junit4:junit4] 2> 27699 T2708 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362265676210/onenodecollection
[junit4:junit4] 2> 27699 T2708 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362265676210/onenodecollection
[junit4:junit4] 2> 27700 T2708 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362265676210/onenodecollection/index
[junit4:junit4] 2> 27700 T2708 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1362265676210/onenodecollection/index
[junit4:junit4] 2> 27702 T2797 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89277443356426246-127.0.0.1:60021_jz%2Fn-n_0000000001) am no longer a leader.
[junit4:junit4] 2> 27702 T2713 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d2d5c99b70006
[junit4:junit4] 2> 27703 T2710 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:54930 which had sessionid 0x13d2d5c99b70006
[junit4:junit4] 2> 27704 T2750 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 27704 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b7000a type:delete cxid:0x53 zxid:0xe3 txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard1 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard1
[junit4:junit4] 2> 27703 T2708 oaz.ZooKeeper.close Session: 0x13d2d5c99b70006 closed
[junit4:junit4] 2> 27704 T2750 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 27705 T2782 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 27704 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70008 type:delete cxid:0x6e zxid:0xe4 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4] 2> 27705 T2750 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 27705 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b7000a type:create cxid:0x54 zxid:0xe5 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 27706 T2750 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 27706 T2766 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 27707 T2782 oasc.ShardLeaderElectionContext.shouldIBeLeader Checking if I should try and be the leader.
[junit4:junit4] 2> 27708 T2782 oasc.ShardLeaderElectionContext.shouldIBeLeader My last published State was Active, it's okay to be the leader.
[junit4:junit4] 2> 27708 T2782 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 27708 T2782 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:32892/jz/n/collection1/
[junit4:junit4] 2> 27709 T2766 oasc.Overseer.start Overseer (id=89277443356426248-127.0.0.1:49462_jz%2Fn-n_0000000002) starting
[junit4:junit4] 2> 27709 T2782 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:32892/jz/n START replicas=[http://127.0.0.1:60021/jz/n/collection1/] nUpdates=100
[junit4:junit4] 2> 27709 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70008 type:create cxid:0x73 zxid:0xe8 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 27710 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70008 type:create cxid:0x74 zxid:0xe9 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 27711 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70008 type:create cxid:0x75 zxid:0xea txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 27712 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b70008 type:create cxid:0x76 zxid:0xeb txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 27712 T2801 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 27713 T2766 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 27713 T2766 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 27715 T2800 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 27716 T2782 oasu.PeerSync.handleResponse WARNING PeerSync: core=collection1 url=http://127.0.0.1:32892/jz/n couldn't connect to http://127.0.0.1:60021/jz/n/collection1/, counting as success
[junit4:junit4] 2> 27716 T2782 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:32892/jz/n DONE. sync succeeded
[junit4:junit4] 2> 27716 T2782 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 27717 T2800 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 27717 T2782 oasc.SyncStrategy.syncToMe http://127.0.0.1:32892/jz/n/collection1/: try and ask http://127.0.0.1:60021/jz/n/collection1/ to sync
[junit4:junit4] 2> 27717 T2800 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "base_url":"http://127.0.0.1:60021/jz/n",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "node_name":"127.0.0.1:60021_jz%2Fn",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 27718 T2782 oasc.SolrException.log SEVERE Sync request error: org.apache.solr.client.solrj.SolrServerException: Server refused connection at: http://127.0.0.1:60021/jz/n/collection1
[junit4:junit4] 2> 27719 T2782 oasc.SyncStrategy.syncToMe http://127.0.0.1:32892/jz/n/collection1/: Sync failed - asking replica (http://127.0.0.1:60021/jz/n/collection1/) to recover.
[junit4:junit4] 2> 27719 T2782 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:32892/jz/n/collection1/
[junit4:junit4] 2> 27719 T2804 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 27719 T2782 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 27723 T2800 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "base_url":"http://127.0.0.1:60021/jz/n",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:60021_jz%2Fn",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"onenodecollectioncore"}
[junit4:junit4] 2> 27723 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b7000a type:create cxid:0x5c zxid:0xef txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 27725 T2782 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 27725 T2782 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 27726 T2804 oasc.SolrException.log SEVERE http://127.0.0.1:32892/jz/n/collection1/: Could not tell a replica to recover:org.apache.solr.client.solrj.SolrServerException: Server refused connection at: http://127.0.0.1:60021/jz/n
[junit4:junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:409)
[junit4:junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181)
[junit4:junit4] 2> at org.apache.solr.cloud.SyncStrategy$1.run(SyncStrategy.java:298)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1121)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:614)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:780)
[junit4:junit4] 2> Caused by: org.apache.http.conn.HttpHostConnectException: Connection to http://127.0.0.1:60021 refused
[junit4:junit4] 2> at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:190)
[junit4:junit4] 2> at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
[junit4:junit4] 2> at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:645)
[junit4:junit4] 2> at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:480)
[junit4:junit4] 2> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
[junit4:junit4] 2> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
[junit4:junit4] 2> at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:784)
[junit4:junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:353)
[junit4:junit4] 2> ... 5 more
[junit4:junit4] 2> Caused by: java.net.ConnectException: Connection refused
[junit4:junit4] 2> at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:369)
[junit4:junit4] 2> at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:230)
[junit4:junit4] 2> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:212)
[junit4:junit4] 2> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:402)
[junit4:junit4] 2> at java.net.Socket.connect(Socket.java:590)
[junit4:junit4] 2> at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:127)
[junit4:junit4] 2> at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
[junit4:junit4] 2> ... 12 more
[junit4:junit4] 2>
[junit4:junit4] 2> 27726 T2708 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/jz/n,null}
[junit4:junit4] 2> 27732 T2766 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 27732 T2782 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 27778 T2708 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 49462
[junit4:junit4] 2> 27779 T2708 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1479167609
[junit4:junit4] 2> 28781 T2708 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 28782 T2708 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 28785 T2708 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@cf8d8b14
[junit4:junit4] 2> 28788 T2708 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3/index
[junit4:junit4] 2> 28789 T2708 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty3
[junit4:junit4] 2> 28792 T2708 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 28793 T2708 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 28794 T2708 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 28795 T2708 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 28796 T2708 C178 P49462 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty2/index
[junit4:junit4] 2> 28797 T2708 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 28798 T2708 oasc.CachingDirectoryFactory.close Releasing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty2/index
[junit4:junit4] 2> 28799 T2708 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty2/index
[junit4:junit4] 2> 28800 T2708 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty2/index
[junit4:junit4] 2> 28801 T2708 oasc.CachingDirectoryFactory.close Closing directory when closing factory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty2
[junit4:junit4] 2> 28801 T2708 oasc.CachingDirectoryFactory.closeDirectory Closing directory:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1362265676210/jetty2
[junit4:junit4] 2> 28803 T2800 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89277443356426248-127.0.0.1:49462_jz%2Fn-n_0000000002) am no longer a leader.
[junit4:junit4] 2> 28803 T2713 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d2d5c99b70008
[junit4:junit4] 2> 28804 T2710 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:54937 which had sessionid 0x13d2d5c99b70008
[junit4:junit4] 2> 28805 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b7000a type:delete cxid:0x64 zxid:0xff txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4] 2> 28805 T2708 oaz.ZooKeeper.close Session: 0x13d2d5c99b70008 closed
[junit4:junit4] 2> 28805 T2782 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 28807 T2782 oasc.Overseer.start Overseer (id=89277443356426250-127.0.0.1:32892_jz%2Fn-n_0000000003) starting
[junit4:junit4] 2> 28808 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b7000a type:create cxid:0x69 zxid:0x101 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 28809 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b7000a type:create cxid:0x6a zxid:0x102 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 28809 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b7000a type:create cxid:0x6b zxid:0x103 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 28810 T2713 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d2d5c99b7000a type:create cxid:0x6c zxid:0x104 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 28811 T2806 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 28812 T2782 oascc.ZkStateReader$3.process Updating live nodes... (1)
[junit4:junit4] 2> 28812 T2782 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 28813 T2805 o
[...truncated too long message...]
nit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4] > at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4:junit4] > at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4:junit4] > at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4] > at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
[junit4:junit4] > at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4] > at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4] > at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4] > at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4] > at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4] > at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4] > at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:780)
[junit4:junit4] 2> 30025 T2708 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 30029 T2707 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 3 leaked thread(s).
[junit4:junit4] 2> 30104 T2766 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 30104 T2766 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 30105 T2766 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 30105 T2766 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 30795 T2711 oazs.SessionTrackerImpl.run SessionTrackerImpl exited loop!
[junit4:junit4] 2> 31203 T2782 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 31203 T2782 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 31204 T2782 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 31205 T2782 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> NOTE: test params are: codec=Lucene42: {range_facet_si=PostingsFormat(name=Memory doPackFST= true), range_facet_l=PostingsFormat(name=Direct), intDefault=PostingsFormat(name=Direct), range_facet_sl=MockVariableIntBlock(baseBlockSize=114), timestamp=PostingsFormat(name=Direct), _version_=PostingsFormat(name=Direct), multiDefault=PostingsFormat(name=Memory doPackFST= true), id=PostingsFormat(name=Memory doPackFST= false)}, docValues:{timestamp=DocValuesFormat(name=Asserting)}, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=ms_MY, timezone=Brazil/East
[junit4:junit4] 2> NOTE: Linux 3.2.0-38-generic x86/IBM Corporation 1.7.0 (32-bit)/cpus=8,threads=3,free=90293600,total=200278016
[junit4:junit4] 2> NOTE: All tests run in this JVM: [BasicDistributedZkTest, TestSolrXMLSerializer, TestIndexSearcher, TestGroupingSearch, DOMUtilTest, TestIndexingPerformance, TestLMDirichletSimilarityFactory, SampleTest, SolrIndexConfigTest, TestSearchPerf, TestComponentsName, SpellingQueryConverterTest, TestElisionMultitermQuery, TestConfig, SolrCoreCheckLockOnStartupTest, TestReload, TestBinaryResponseWriter, TestDefaultSimilarityFactory, ShardRoutingCustomTest, TestPHPSerializedResponseWriter, TestStressReorder, TestArbitraryIndexDir, PrimUtilsTest, TestSolrIndexConfig, QueryElevationComponentTest, TestWordDelimiterFilterFactory, SolrCmdDistributorTest, SimplePostToolTest, TestPropInject, HardAutoCommitTest, OutputWriterTest, SpatialFilterTest, DebugComponentTest, LeaderElectionIntegrationTest, SpellCheckCollatorTest, TestOmitPositions, TestCodecSupport, DirectSolrConnectionTest, TestRTGBase, ReturnFieldsTest, FileUtilsTest, TestSurroundQueryParser, TestUpdate, TestBinaryField, IndexBasedSpellCheckerTest, SpellCheckComponentTest, XmlUpdateRequestHandlerTest, ScriptEngineTest, DistributedTermsComponentTest, ZkCLITest, LoggingHandlerTest, PluginInfoTest, TestDocumentBuilder, ClusterStateTest, PreAnalyzedFieldTest, ClusterStateUpdateTest, DocValuesMultiTest, WordBreakSolrSpellCheckerTest, TestSolrJ, DateMathParserTest, SpellPossibilityIteratorTest, TestSolr4Spatial, DefaultValueUpdateProcessorTest, OverseerTest, FullSolrCloudDistribCmdsTest, DistributedSpellCheckComponentTest, TestRealTimeGet, TestRandomFaceting, TestRangeQuery, DirectUpdateHandlerTest, SimpleFacetsTest, TestCoreContainer, LukeRequestHandlerTest, TestFiltering, FileBasedSpellCheckerTest, PeerSyncTest, BasicFunctionalityTest, TestTrie, CacheHeaderTest, IndexSchemaTest, TermsComponentTest, XsltUpdateRequestHandlerTest, PingRequestHandlerTest, DocumentAnalysisRequestHandlerTest, DirectUpdateHandlerOptimizeTest, TestValueSourceCache, JSONWriterTest, SearchHandlerTest, SuggesterFSTTest, SuggesterTest, ShowFileRequestHandlerTest, PrimitiveFieldTypeTest, FieldMutatingUpdateProcessorTest, TestSolrDeletionPolicy2, TestPropInjectDefaults, TestXIncludeConfig, TestLFUCache, TestQuerySenderListener, NumericFieldsTest, TestJmxMonitoredMap, TestBM25SimilarityFactory, TestPluginEnable, TestDFRSimilarityFactory, TimeZoneUtilsTest, OpenExchangeRatesOrgProviderTest, TestSystemIdResolver, TestBadConfig, RAMDirectoryFactoryTest, SystemInfoHandlerTest, TestUtils, AnalysisAfterCoreReloadTest, TestRandomDVFaceting, TestCharFilters, BasicDistributedZk2Test]
[junit4:junit4] Completed on J0 in 31.30s, 1 test, 1 failure <<< FAILURES!
[...truncated 81 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:381: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:374: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:183: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:449: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1207: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:870: There were test failures: 260 suites, 1095 tests, 1 failure, 10 ignored (4 assumptions)
Total time: 46 minutes 23 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Description set: Java: 32bit/ibm-j9-jdk7
Email was triggered for: Failure
Sending email for trigger: Failure