You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2013/03/24 09:48:10 UTC
[JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b82) - Build #
4803 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/4803/
Java: 32bit/jdk1.8.0-ea-b82 -server -XX:+UseG1GC -XX:MarkStackSize=256K
2 tests failed.
REGRESSION: org.apache.solr.cloud.BasicDistributedZk2Test.testDistribSearch
Error Message:
Server at http://127.0.0.1:51388/onenodecollectioncore returned non ok status:404, message:Can not find: /onenodecollectioncore/update
Stack Trace:
org.apache.solr.common.SolrException: Server at http://127.0.0.1:51388/onenodecollectioncore returned non ok status:404, message:Can not find: /onenodecollectioncore/update
at __randomizedtesting.SeedInfo.seed([9753738F1061972F:16B5FD97673EF713]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:375)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181)
at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:116)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:102)
at org.apache.solr.cloud.BasicDistributedZk2Test.testNodeWithoutCollectionForwarding(BasicDistributedZk2Test.java:197)
at org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:89)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:487)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
REGRESSION: org.apache.solr.cloud.UnloadDistributedZkTest.testDistribSearch
Error Message:
Still found shard2 in collection test_unload_shard_and_collection
Stack Trace:
java.lang.AssertionError: Still found shard2 in collection test_unload_shard_and_collection
at __randomizedtesting.SeedInfo.seed([9753738F1061972F:16B5FD97673EF713]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.UnloadDistributedZkTest.testUnloadShardAndCollection(UnloadDistributedZkTest.java:127)
at org.apache.solr.cloud.UnloadDistributedZkTest.doTest(UnloadDistributedZkTest.java:77)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:487)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Build Log:
[...truncated 9416 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZk2Test
[junit4:junit4] 2> 0 T1671 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4] 2> 3 T1671 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1364114080320
[junit4:junit4] 2> 4 T1671 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 4 T1672 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 5 T1672 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4] 2> 5 T1672 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4] 2> 5 T1672 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4] 2> 6 T1672 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4] 2> 6 T1672 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-1364114080320/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4] 2> 104 T1671 oasc.ZkTestServer.run start zk server on port:51859
[junit4:junit4] 2> 105 T1671 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:51859 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@1f65271
[junit4:junit4] 2> 106 T1671 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 106 T1677 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:51859. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 107 T1677 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:51859, initiating session
[junit4:junit4] 2> 107 T1673 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:34969
[junit4:junit4] 2> 107 T1673 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:34969
[junit4:junit4] 2> 108 T1675 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4] 2> 174 T1675 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9b8902430000 with negotiated timeout 10000 for client /127.0.0.1:34969
[junit4:junit4] 2> 174 T1677 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:51859, sessionid = 0x13d9b8902430000, negotiated timeout = 10000
[junit4:junit4] 2> 175 T1678 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f65271 name:ZooKeeperConnection Watcher:127.0.0.1:51859 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 175 T1671 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 175 T1671 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 180 T1676 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d9b8902430000
[junit4:junit4] 2> 181 T1678 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 181 T1673 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:34969 which had sessionid 0x13d9b8902430000
[junit4:junit4] 2> 181 T1671 oaz.ZooKeeper.close Session: 0x13d9b8902430000 closed
[junit4:junit4] 2> 182 T1671 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:51859/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@131bd7f
[junit4:junit4] 2> 183 T1679 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:51859. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 183 T1671 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 183 T1679 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:51859, initiating session
[junit4:junit4] 2> 184 T1673 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:34970
[junit4:junit4] 2> 184 T1673 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:34970
[junit4:junit4] 2> 185 T1675 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9b8902430001 with negotiated timeout 10000 for client /127.0.0.1:34970
[junit4:junit4] 2> 185 T1679 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:51859, sessionid = 0x13d9b8902430001, negotiated timeout = 10000
[junit4:junit4] 2> 186 T1680 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@131bd7f name:ZooKeeperConnection Watcher:127.0.0.1:51859/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 186 T1671 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 186 T1671 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 190 T1671 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 193 T1671 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 196 T1671 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 199 T1671 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 200 T1671 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 206 T1671 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 207 T1671 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 212 T1671 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 213 T1671 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 216 T1671 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 217 T1671 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 220 T1671 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 221 T1671 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 225 T1671 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 226 T1671 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 230 T1671 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 231 T1671 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 235 T1671 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 236 T1671 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 240 T1671 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 241 T1671 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 244 T1676 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d9b8902430001
[junit4:junit4] 2> 245 T1673 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:34970 which had sessionid 0x13d9b8902430001
[junit4:junit4] 2> 245 T1680 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 245 T1671 oaz.ZooKeeper.close Session: 0x13d9b8902430001 closed
[junit4:junit4] 2> 312 T1671 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 317 T1671 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:53297
[junit4:junit4] 2> 318 T1671 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 318 T1671 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 318 T1671 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1364114080562
[junit4:junit4] 2> 319 T1671 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1364114080562/solr.xml
[junit4:junit4] 2> 319 T1671 oasc.CoreContainer.<init> New CoreContainer 11595844
[junit4:junit4] 2> 319 T1671 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1364114080562/'
[junit4:junit4] 2> 320 T1671 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1364114080562/'
[junit4:junit4] 2> 335 T1671 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 336 T1671 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 336 T1671 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 336 T1671 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 337 T1671 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 337 T1671 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 337 T1671 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 338 T1671 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 338 T1671 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 338 T1671 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 344 T1671 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 353 T1671 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:51859/solr
[junit4:junit4] 2> 353 T1671 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 353 T1671 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:51859 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@18e2d31
[junit4:junit4] 2> 355 T1690 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:51859. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 354 T1671 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 356 T1690 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:51859, initiating session
[junit4:junit4] 2> 356 T1673 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:34971
[junit4:junit4] 2> 356 T1673 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:34971
[junit4:junit4] 2> 357 T1675 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9b8902430002 with negotiated timeout 20000 for client /127.0.0.1:34971
[junit4:junit4] 2> 357 T1690 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:51859, sessionid = 0x13d9b8902430002, negotiated timeout = 20000
[junit4:junit4] 2> 358 T1691 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@18e2d31 name:ZooKeeperConnection Watcher:127.0.0.1:51859 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 358 T1671 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 360 T1676 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d9b8902430002
[junit4:junit4] 2> 361 T1691 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 361 T1673 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:34971 which had sessionid 0x13d9b8902430002
[junit4:junit4] 2> 361 T1671 oaz.ZooKeeper.close Session: 0x13d9b8902430002 closed
[junit4:junit4] 2> 362 T1671 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 366 T1671 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:51859/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@1a8d98e
[junit4:junit4] 2> 367 T1692 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:51859. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 367 T1671 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 367 T1673 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:34972
[junit4:junit4] 2> 367 T1692 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:51859, initiating session
[junit4:junit4] 2> 368 T1673 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:34972
[junit4:junit4] 2> 369 T1675 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9b8902430003 with negotiated timeout 20000 for client /127.0.0.1:34972
[junit4:junit4] 2> 369 T1692 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:51859, sessionid = 0x13d9b8902430003, negotiated timeout = 20000
[junit4:junit4] 2> 370 T1693 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a8d98e name:ZooKeeperConnection Watcher:127.0.0.1:51859/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 370 T1671 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 372 T1671 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 375 T1676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9b8902430003 type:create cxid:0x7 zxid:0x1b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 377 T1671 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 381 T1671 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 383 T1671 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53297_
[junit4:junit4] 2> 384 T1676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9b8902430003 type:delete cxid:0x12 zxid:0x1e txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:53297_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:53297_
[junit4:junit4] 2> 385 T1671 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:53297_
[junit4:junit4] 2> 390 T1671 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 397 T1676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9b8902430003 type:delete cxid:0x20 zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4] 2> 398 T1671 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 402 T1671 oasc.Overseer.start Overseer (id=89398580368048131-127.0.0.1:53297_-n_0000000000) starting
[junit4:junit4] 2> 403 T1676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9b8902430003 type:create cxid:0x25 zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 404 T1676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9b8902430003 type:create cxid:0x26 zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 406 T1676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9b8902430003 type:create cxid:0x28 zxid:0x27 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 408 T1671 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 412 T1676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9b8902430003 type:create cxid:0x2e zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 414 T1695 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 415 T1671 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 418 T1671 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 421 T1671 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 425 T1694 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 429 T1696 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1364114080562/collection1
[junit4:junit4] 2> 430 T1696 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 431 T1696 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 431 T1696 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 433 T1696 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1364114080562/collection1/'
[junit4:junit4] 2> 434 T1696 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1364114080562/collection1/lib/README' to classloader
[junit4:junit4] 2> 434 T1696 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1364114080562/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 464 T1696 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4] 2> 491 T1696 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 492 T1696 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 496 T1696 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 821 T1696 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 826 T1696 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 828 T1696 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 844 T1696 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 848 T1696 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 851 T1696 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 853 T1696 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 854 T1696 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 854 T1696 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 856 T1696 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 856 T1696 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 857 T1696 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 857 T1696 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1364114080562/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1364114080320/control/data/
[junit4:junit4] 2> 857 T1696 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7d3c85
[junit4:junit4] 2> 858 T1696 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 859 T1696 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1364114080320/control/data
[junit4:junit4] 2> 859 T1696 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1364114080320/control/data/index/
[junit4:junit4] 2> 859 T1696 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1364114080320/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 860 T1696 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1364114080320/control/data/index
[junit4:junit4] 2> 862 T1696 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1c196bd lockFactory=org.apache.lucene.store.NativeFSLockFactory@ef49bc),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 863 T1696 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 865 T1696 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 866 T1696 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 866 T1696 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 867 T1696 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 868 T1696 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 868 T1696 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 869 T1696 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 869 T1696 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 870 T1696 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 874 T1696 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 878 T1696 oass.SolrIndexSearcher.<init> Opening Searcher@ebe25a main
[junit4:junit4] 2> 879 T1696 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1364114080320/control/data/tlog
[junit4:junit4] 2> 880 T1696 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 880 T1696 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 884 T1697 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@ebe25a main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 886 T1696 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 886 T1696 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1931 T1694 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1932 T1694 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:53297_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:53297",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 1933 T1694 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 1933 T1694 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 1938 T1693 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 2888 T1696 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 2888 T1696 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:53297 collection:control_collection shard:shard1
[junit4:junit4] 2> 2889 T1696 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 2899 T1676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9b8902430003 type:delete cxid:0x82 zxid:0x35 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4] 2> 2900 T1696 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 2900 T1676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9b8902430003 type:create cxid:0x83 zxid:0x36 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 2903 T1696 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 2903 T1696 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 2904 T1696 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:53297/collection1/
[junit4:junit4] 2> 2904 T1696 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 2904 T1696 oasc.SyncStrategy.syncToMe http://127.0.0.1:53297/collection1/ has no replicas
[junit4:junit4] 2> 2905 T1696 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:53297/collection1/
[junit4:junit4] 2> 2905 T1696 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 2910 T1676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9b8902430003 type:create cxid:0x8e zxid:0x3a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 3445 T1694 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 3457 T1693 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 3465 T1696 oasc.ZkController.register We are http://127.0.0.1:53297/collection1/ and leader is http://127.0.0.1:53297/collection1/
[junit4:junit4] 2> 3465 T1696 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:53297
[junit4:junit4] 2> 3465 T1696 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 3465 T1696 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 3466 T1696 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 3468 T1696 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 3470 T1671 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 3470 T1671 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 3471 T1671 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 3478 T1671 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 3479 T1671 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:51859/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@1fa26e6
[junit4:junit4] 2> 3481 T1699 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:51859. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 3480 T1671 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 3481 T1699 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:51859, initiating session
[junit4:junit4] 2> 3481 T1673 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:34973
[junit4:junit4] 2> 3482 T1673 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:34973
[junit4:junit4] 2> 3483 T1675 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9b8902430004 with negotiated timeout 10000 for client /127.0.0.1:34973
[junit4:junit4] 2> 3483 T1699 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:51859, sessionid = 0x13d9b8902430004, negotiated timeout = 10000
[junit4:junit4] 2> 3484 T1700 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1fa26e6 name:ZooKeeperConnection Watcher:127.0.0.1:51859/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 3484 T1671 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 3486 T1671 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 3490 T1671 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 3555 T1671 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 3558 T1671 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:34938
[junit4:junit4] 2> 3560 T1671 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 3561 T1671 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 3561 T1671 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1364114083806
[junit4:junit4] 2> 3562 T1671 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1364114083806/solr.xml
[junit4:junit4] 2> 3563 T1671 oasc.CoreContainer.<init> New CoreContainer 20209315
[junit4:junit4] 2> 3564 T1671 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1364114083806/'
[junit4:junit4] 2> 3564 T1671 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1364114083806/'
[junit4:junit4] 2> 3586 T1671 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 3586 T1671 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 3587 T1671 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 3587 T1671 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 3587 T1671 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 3588 T1671 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 3588 T1671 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 3588 T1671 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 3589 T1671 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 3589 T1671 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 3594 T1671 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 3603 T1671 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:51859/solr
[junit4:junit4] 2> 3603 T1671 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 3604 T1671 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:51859 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@1d1da0c
[junit4:junit4] 2> 3605 T1710 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:51859. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 3605 T1671 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 3606 T1710 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:51859, initiating session
[junit4:junit4] 2> 3606 T1673 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:34974
[junit4:junit4] 2> 3606 T1673 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:34974
[junit4:junit4] 2> 3607 T1675 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9b8902430005 with negotiated timeout 20000 for client /127.0.0.1:34974
[junit4:junit4] 2> 3607 T1710 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:51859, sessionid = 0x13d9b8902430005, negotiated timeout = 20000
[junit4:junit4] 2> 3607 T1711 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1d1da0c name:ZooKeeperConnection Watcher:127.0.0.1:51859 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 3607 T1671 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 3609 T1676 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d9b8902430005
[junit4:junit4] 2> 3609 T1711 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 3609 T1673 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:34974 which had sessionid 0x13d9b8902430005
[junit4:junit4] 2> 3609 T1671 oaz.ZooKeeper.close Session: 0x13d9b8902430005 closed
[junit4:junit4] 2> 3610 T1671 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 3616 T1671 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:51859/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@11ef52b
[junit4:junit4] 2> 3617 T1712 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:51859. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 3617 T1671 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 3618 T1712 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:51859, initiating session
[junit4:junit4] 2> 3618 T1673 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:34975
[junit4:junit4] 2> 3619 T1673 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:34975
[junit4:junit4] 2> 3619 T1675 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9b8902430006 with negotiated timeout 20000 for client /127.0.0.1:34975
[junit4:junit4] 2> 3620 T1712 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:51859, sessionid = 0x13d9b8902430006, negotiated timeout = 20000
[junit4:junit4] 2> 3620 T1713 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@11ef52b name:ZooKeeperConnection Watcher:127.0.0.1:51859/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 3620 T1671 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 3622 T1676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9b8902430006 type:create cxid:0x1 zxid:0x48 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 3623 T1676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9b8902430006 type:create cxid:0x3 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 3627 T1671 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 4631 T1671 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34938_
[junit4:junit4] 2> 4633 T1676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9b8902430006 type:delete cxid:0xd zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:34938_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:34938_
[junit4:junit4] 2> 4634 T1671 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:34938_
[junit4:junit4] 2> 4637 T1693 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 4638 T1700 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 4638 T1713 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 4640 T1693 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 4647 T1714 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1364114083806/collection1
[junit4:junit4] 2> 4648 T1714 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 4649 T1714 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 4649 T1714 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 4651 T1714 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1364114083806/collection1/'
[junit4:junit4] 2> 4652 T1714 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1364114083806/collection1/lib/README' to classloader
[junit4:junit4] 2> 4652 T1714 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1364114083806/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 4684 T1714 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4] 2> 4710 T1714 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 4711 T1714 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 4714 T1714 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 4964 T1694 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 4964 T1694 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:53297_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:53297",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 4967 T1693 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 4967 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 4967 T1700 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 5013 T1714 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 5019 T1714 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 5021 T1714 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 5048 T1714 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 5051 T1714 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 5055 T1714 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 5057 T1714 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 5057 T1714 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 5058 T1714 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 5059 T1714 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 5059 T1714 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 5060 T1714 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 5060 T1714 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1364114083806/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1364114080320/jetty1/
[junit4:junit4] 2> 5060 T1714 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7d3c85
[junit4:junit4] 2> 5061 T1714 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 5061 T1714 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1364114080320/jetty1
[junit4:junit4] 2> 5062 T1714 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1364114080320/jetty1/index/
[junit4:junit4] 2> 5062 T1714 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1364114080320/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 5062 T1714 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1364114080320/jetty1/index
[junit4:junit4] 2> 5064 T1714 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1a527b5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@10e5d6),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 5064 T1714 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 5066 T1714 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 5067 T1714 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 5067 T1714 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 5068 T1714 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 5069 T1714 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 5069 T1714 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 5069 T1714 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 5070 T1714 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 5070 T1714 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 5074 T1714 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 5077 T1714 oass.SolrIndexSearcher.<init> Opening Searcher@1af5bfa main
[junit4:junit4] 2> 5078 T1714 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1364114080320/jetty1/tlog
[junit4:junit4] 2> 5079 T1714 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 5079 T1714 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 5085 T1715 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1af5bfa main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 5087 T1714 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 5088 T1714 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 6472 T1694 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 6473 T1694 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:34938_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34938",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 6473 T1694 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4] 2> 6474 T1694 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4] 2> 6481 T1693 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 6481 T1700 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 6481 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 7090 T1714 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 7090 T1714 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:34938 collection:collection1 shard:shard2
[junit4:junit4] 2> 7091 T1714 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 7103 T1676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9b8902430006 type:delete cxid:0x4a zxid:0x5a txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4] 2> 7104 T1714 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 7105 T1676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9b8902430006 type:create cxid:0x4b zxid:0x5b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 7108 T1714 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 7109 T1714 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 7109 T1714 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:34938/collection1/
[junit4:junit4] 2> 7109 T1714 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 7110 T1714 oasc.SyncStrategy.syncToMe http://127.0.0.1:34938/collection1/ has no replicas
[junit4:junit4] 2> 7110 T1714 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:34938/collection1/
[junit4:junit4] 2> 7110 T1714 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 7119 T1676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9b8902430006 type:create cxid:0x56 zxid:0x5f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 7988 T1694 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 7998 T1693 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 7998 T1700 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 7998 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 8026 T1714 oasc.ZkController.register We are http://127.0.0.1:34938/collection1/ and leader is http://127.0.0.1:34938/collection1/
[junit4:junit4] 2> 8027 T1714 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:34938
[junit4:junit4] 2> 8027 T1714 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 8027 T1714 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 8027 T1714 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 8030 T1714 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 8031 T1671 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 8032 T1671 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 8033 T1671 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 8098 T1671 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 8102 T1671 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51388
[junit4:junit4] 2> 8103 T1671 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 8103 T1671 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 8103 T1671 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1364114088356
[junit4:junit4] 2> 8104 T1671 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1364114088356/solr.xml
[junit4:junit4] 2> 8104 T1671 oasc.CoreContainer.<init> New CoreContainer 16355626
[junit4:junit4] 2> 8105 T1671 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1364114088356/'
[junit4:junit4] 2> 8106 T1671 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1364114088356/'
[junit4:junit4] 2> 8125 T1671 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 8125 T1671 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 8125 T1671 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 8126 T1671 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 8126 T1671 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 8126 T1671 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 8127 T1671 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 8127 T1671 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 8127 T1671 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 8128 T1671 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 8133 T1671 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 8142 T1671 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:51859/solr
[junit4:junit4] 2> 8142 T1671 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 8142 T1671 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:51859 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@1961a12
[junit4:junit4] 2> 8143 T1671 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 8144 T1726 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:51859. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 8144 T1726 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:51859, initiating session
[junit4:junit4] 2> 8144 T1673 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:34976
[junit4:junit4] 2> 8145 T1673 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:34976
[junit4:junit4] 2> 8146 T1675 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9b8902430007 with negotiated timeout 20000 for client /127.0.0.1:34976
[junit4:junit4] 2> 8146 T1726 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:51859, sessionid = 0x13d9b8902430007, negotiated timeout = 20000
[junit4:junit4] 2> 8146 T1727 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1961a12 name:ZooKeeperConnection Watcher:127.0.0.1:51859 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 8147 T1671 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 8148 T1676 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d9b8902430007
[junit4:junit4] 2> 8149 T1727 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 8149 T1673 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:34976 which had sessionid 0x13d9b8902430007
[junit4:junit4] 2> 8149 T1671 oaz.ZooKeeper.close Session: 0x13d9b8902430007 closed
[junit4:junit4] 2> 8149 T1671 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 8152 T1671 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:51859/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@147b2c
[junit4:junit4] 2> 8153 T1728 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:51859. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 8153 T1671 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 8153 T1728 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:51859, initiating session
[junit4:junit4] 2> 8153 T1673 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:34977
[junit4:junit4] 2> 8154 T1673 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:34977
[junit4:junit4] 2> 8155 T1675 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9b8902430008 with negotiated timeout 20000 for client /127.0.0.1:34977
[junit4:junit4] 2> 8155 T1728 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:51859, sessionid = 0x13d9b8902430008, negotiated timeout = 20000
[junit4:junit4] 2> 8155 T1729 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@147b2c name:ZooKeeperConnection Watcher:127.0.0.1:51859/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 8155 T1671 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 8156 T1676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9b8902430008 type:create cxid:0x1 zxid:0x6c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 8158 T1676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9b8902430008 type:create cxid:0x3 zxid:0x6d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 8160 T1671 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 9163 T1671 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51388_
[junit4:junit4] 2> 9165 T1676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9b8902430008 type:delete cxid:0xd zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:51388_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:51388_
[junit4:junit4] 2> 9167 T1671 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51388_
[junit4:junit4] 2> 9170 T1700 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 9171 T1713 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9171 T1729 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9171 T1693 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9171 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9172 T1693 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9173 T1700 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9180 T1730 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1364114088356/collection1
[junit4:junit4] 2> 9181 T1730 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 9182 T1730 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 9182 T1730 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 9184 T1730 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1364114088356/collection1/'
[junit4:junit4] 2> 9185 T1730 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1364114088356/collection1/lib/README' to classloader
[junit4:junit4] 2> 9185 T1730 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1364114088356/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 9220 T1730 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4] 2> 9246 T1730 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 9248 T1730 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 9251 T1730 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 9505 T1694 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 9506 T1694 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:34938_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34938",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 9510 T1693 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9510 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9510 T1700 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9510 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9550 T1730 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 9556 T1730 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 9558 T1730 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 9573 T1730 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 9576 T1730 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 9579 T1730 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 9580 T1730 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 9581 T1730 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 9581 T1730 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 9582 T1730 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 9582 T1730 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 9582 T1730 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 9582 T1730 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1364114088356/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1364114080320/jetty2/
[junit4:junit4] 2> 9583 T1730 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7d3c85
[junit4:junit4] 2> 9583 T1730 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 9583 T1730 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1364114080320/jetty2
[junit4:junit4] 2> 9584 T1730 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1364114080320/jetty2/index/
[junit4:junit4] 2> 9584 T1730 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1364114080320/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 9584 T1730 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1364114080320/jetty2/index
[junit4:junit4] 2> 9585 T1730 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6915ec lockFactory=org.apache.lucene.store.NativeFSLockFactory@1715ac8),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 9585 T1730 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 9587 T1730 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 9587 T1730 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 9587 T1730 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 9588 T1730 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 9588 T1730 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 9588 T1730 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 9589 T1730 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 9589 T1730 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 9589 T1730 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 9591 T1730 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 9593 T1730 oass.SolrIndexSearcher.<init> Opening Searcher@506675 main
[junit4:junit4] 2> 9594 T1730 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1364114080320/jetty2/tlog
[junit4:junit4] 2> 9594 T1730 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 9594 T1730 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 9597 T1731 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@506675 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 9599 T1730 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 9599 T1730 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 11015 T1694 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 11017 T1694 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51388_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51388",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 11017 T1694 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 11017 T1694 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 11024 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 11024 T1700 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 11024 T1693 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 11024 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 11601 T1730 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 11601 T1730 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:51388 collection:collection1 shard:shard1
[junit4:junit4] 2> 11602 T1730 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 11611 T1676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9b8902430008 type:delete cxid:0x49 zxid:0x7d txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard1 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard1
[junit4:junit4] 2> 11612 T1730 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 11613 T1676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9b8902430008 type:create cxid:0x4a zxid:0x7e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 11617 T1730 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 11618 T1730 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 11618 T1730 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:51388/collection1/
[junit4:junit4] 2> 11618 T1730 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 11619 T1730 oasc.SyncStrategy.syncToMe http://127.0.0.1:51388/collection1/ has no replicas
[junit4:junit4] 2> 11619 T1730 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:51388/collection1/
[junit4:junit4] 2> 11619 T1730 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 11625 T1676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9b8902430008 type:create cxid:0x54 zxid:0x81 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 12530 T1694 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 12542 T1693 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 12542 T1700 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 12542 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 12542 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 12581 T1730 oasc.ZkController.register We are http://127.0.0.1:51388/collection1/ and leader is http://127.0.0.1:51388/collection1/
[junit4:junit4] 2> 12581 T1730 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:51388
[junit4:junit4] 2> 12582 T1730 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 12582 T1730 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 12582 T1730 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 12585 T1730 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 12587 T1671 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 12587 T1671 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 12588 T1671 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 12656 T1671 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 12658 T1671 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:37806
[junit4:junit4] 2> 12660 T1671 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 12661 T1671 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 12661 T1671 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1364114092911
[junit4:junit4] 2> 12662 T1671 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1364114092911/solr.xml
[junit4:junit4] 2> 12662 T1671 oasc.CoreContainer.<init> New CoreContainer 31500494
[junit4:junit4] 2> 12663 T1671 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1364114092911/'
[junit4:junit4] 2> 12663 T1671 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1364114092911/'
[junit4:junit4] 2> 12686 T1671 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 12686 T1671 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 12686 T1671 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 12687 T1671 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 12687 T1671 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 12687 T1671 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 12688 T1671 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 12688 T1671 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 12688 T1671 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 12688 T1671 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 12694 T1671 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 12714 T1671 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:51859/solr
[junit4:junit4] 2> 12714 T1671 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 12715 T1671 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:51859 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@1140df8
[junit4:junit4] 2> 12716 T1742 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:51859. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 12715 T1671 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 12716 T1742 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:51859, initiating session
[junit4:junit4] 2> 12717 T1673 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:34979
[junit4:junit4] 2> 12717 T1673 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:34979
[junit4:junit4] 2> 12718 T1675 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9b8902430009 with negotiated timeout 20000 for client /127.0.0.1:34979
[junit4:junit4] 2> 12718 T1742 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:51859, sessionid = 0x13d9b8902430009, negotiated timeout = 20000
[junit4:junit4] 2> 12719 T1743 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1140df8 name:ZooKeeperConnection Watcher:127.0.0.1:51859 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 12719 T1671 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 12721 T1676 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d9b8902430009
[junit4:junit4] 2> 12722 T1743 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 12722 T1673 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:34979 which had sessionid 0x13d9b8902430009
[junit4:junit4] 2> 12722 T1671 oaz.ZooKeeper.close Session: 0x13d9b8902430009 closed
[junit4:junit4] 2> 12722 T1671 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 12725 T1671 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:51859/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@1e86092
[junit4:junit4] 2> 12726 T1744 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:51859. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 12726 T1671 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 12727 T1744 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:51859, initiating session
[junit4:junit4] 2> 12727 T1673 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:34980
[junit4:junit4] 2> 12727 T1673 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:34980
[junit4:junit4] 2> 12728 T1675 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9b890243000a with negotiated timeout 20000 for client /127.0.0.1:34980
[junit4:junit4] 2> 12728 T1744 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:51859, sessionid = 0x13d9b890243000a, negotiated timeout = 20000
[junit4:junit4] 2> 12729 T1745 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1e86092 name:ZooKeeperConnection Watcher:127.0.0.1:51859/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 12729 T1671 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 12730 T1676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9b890243000a type:create cxid:0x1 zxid:0x8e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 12732 T1676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9b890243000a type:create cxid:0x3 zxid:0x8f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 12735 T1671 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 13740 T1671 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37806_
[junit4:junit4] 2> 13741 T1676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9b890243000a type:delete cxid:0xd zxid:0x90 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:37806_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:37806_
[junit4:junit4] 2> 13742 T1671 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:37806_
[junit4:junit4] 2> 13745 T1693 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 13745 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 13745 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 13746 T1745 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 13746 T1700 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 13747 T1700 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 13748 T1729 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 13748 T1693 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 13749 T1713 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 13756 T1746 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1364114092911/collection1
[junit4:junit4] 2> 13757 T1746 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 13758 T1746 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 13758 T1746 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 13760 T1746 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1364114092911/collection1/'
[junit4:junit4] 2> 13761 T1746 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1364114092911/collection1/lib/README' to classloader
[junit4:junit4] 2> 13761 T1746 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1364114092911/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 13788 T1746 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4] 2> 13814 T1746 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 13815 T1746 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 13819 T1746 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 14050 T1694 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 14051 T1694 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51388_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51388",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 14054 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14054 T1693 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14054 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14054 T1700 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14054 T1745 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14111 T1746 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 14117 T1746 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 14119 T1746 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 14134 T1746 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 14137 T1746 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 14141 T1746 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 14142 T1746 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 14143 T1746 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 14143 T1746 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 14145 T1746 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 14145 T1746 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 14145 T1746 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 14145 T1746 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1364114092911/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1364114080320/jetty3/
[junit4:junit4] 2> 14146 T1746 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7d3c85
[junit4:junit4] 2> 14146 T1746 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 14147 T1746 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1364114080320/jetty3
[junit4:junit4] 2> 14147 T1746 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1364114080320/jetty3/index/
[junit4:junit4] 2> 14147 T1746 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1364114080320/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 14148 T1746 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1364114080320/jetty3/index
[junit4:junit4] 2> 14149 T1746 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1dcf5c6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@f303f5),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 14149 T1746 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 14151 T1746 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 14152 T1746 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 14152 T1746 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 14153 T1746 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 14154 T1746 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 14154 T1746 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 14154 T1746 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 14155 T1746 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 14155 T1746 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 14158 T1746 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 14162 T1746 oass.SolrIndexSearcher.<init> Opening Searcher@165306a main
[junit4:junit4] 2> 14162 T1746 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1364114080320/jetty3/tlog
[junit4:junit4] 2> 14163 T1746 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 14163 T1746 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 14169 T1747 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@165306a main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 14172 T1746 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 14172 T1746 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 15559 T1694 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 15561 T1694 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:37806_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:37806",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 15561 T1694 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 15561 T1694 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4] 2> 15569 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 15569 T1700 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 15569 T1693 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 15569 T1745 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 15569 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 16174 T1746 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 16174 T1746 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:37806 collection:collection1 shard:shard2
[junit4:junit4] 2> 16180 T1746 oasc.ZkController.register We are http://127.0.0.1:37806/collection1/ and leader is http://127.0.0.1:34938/collection1/
[junit4:junit4] 2> 16180 T1746 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:37806
[junit4:junit4] 2> 16180 T1746 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 16181 T1746 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C93 name=collection1 org.apache.solr.core.SolrCore@6b4200 url=http://127.0.0.1:37806/collection1 node=127.0.0.1:37806_ C93_STATE=coll:collection1 core:collection1 props:{state=down, collection=collection1, node_name=127.0.0.1:37806_, base_url=http://127.0.0.1:37806, core=collection1}
[junit4:junit4] 2> 16181 T1748 C93 P37806 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 16183 T1748 C93 P37806 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 16183 T1746 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 16183 T1748 C93 P37806 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 16183 T1748 C93 P37806 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 16185 T1671 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 16185 T1748 C93 P37806 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 16186 T1671 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 16186 T1671 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 16258 T1671 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 16260 T1671 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49725
[junit4:junit4] 2> 16261 T1671 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 16262 T1671 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 16263 T1671 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1364114096508
[junit4:junit4] 2> 16264 T1671 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1364114096508/solr.xml
[junit4:junit4] 2> 16264 T1671 oasc.CoreContainer.<init> New CoreContainer 9994875
[junit4:junit4] 2> 16265 T1671 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1364114096508/'
[junit4:junit4] 2> 16266 T1671 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1364114096508/'
[junit4:junit4] 2> 16285 T1671 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 16285 T1671 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 16285 T1671 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 16286 T1671 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 16286 T1671 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 16286 T1671 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 16287 T1671 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 16287 T1671 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 16287 T1671 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 16288 T1671 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 16293 T1671 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 16302 T1671 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:51859/solr
[junit4:junit4] 2> 16302 T1671 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 16303 T1671 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:51859 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@142758e
[junit4:junit4] 2> 16304 T1759 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:51859. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 16304 T1671 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 16304 T1759 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:51859, initiating session
[junit4:junit4] 2> 16305 T1673 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:34985
[junit4:junit4] 2> 16305 T1673 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:34985
[junit4:junit4] 2> 16306 T1675 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9b890243000b with negotiated timeout 20000 for client /127.0.0.1:34985
[junit4:junit4] 2> 16306 T1759 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:51859, sessionid = 0x13d9b890243000b, negotiated timeout = 20000
[junit4:junit4] 2> 16306 T1760 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@142758e name:ZooKeeperConnection Watcher:127.0.0.1:51859 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 16307 T1671 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 16308 T1676 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d9b890243000b
[junit4:junit4] 2> 16308 T1760 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 16309 T1673 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:34985 which had sessionid 0x13d9b890243000b
[junit4:junit4] 2> 16308 T1671 oaz.ZooKeeper.close Session: 0x13d9b890243000b closed
[junit4:junit4] 2> 16309 T1671 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 16312 T1671 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:51859/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@1a511e6
[junit4:junit4] 2> 16312 T1761 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:51859. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 16312 T1671 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 16313 T1761 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:51859, initiating session
[junit4:junit4] 2> 16313 T1673 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:34986
[junit4:junit4] 2> 16313 T1673 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:34986
[junit4:junit4] 2> 16314 T1675 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9b890243000c with negotiated timeout 20000 for client /127.0.0.1:34986
[junit4:junit4] 2> 16314 T1761 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:51859, sessionid = 0x13d9b890243000c, negotiated timeout = 20000
[junit4:junit4] 2> 16315 T1762 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a511e6 name:ZooKeeperConnection Watcher:127.0.0.1:51859/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 16315 T1671 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 16316 T1676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9b890243000c type:create cxid:0x1 zxid:0xa1 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 16317 T1676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9b890243000c type:create cxid:0x3 zxid:0xa2 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 16319 T1671 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 17076 T1694 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 17077 T1694 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:37806_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:37806",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 17083 T1745 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17083 T1700 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17083 T1762 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17083 T1693 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17083 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17083 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17194 T1706 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {state=recovering&coreNodeName=4&action=PREPRECOVERY&wt=javabin&version=2&checkLive=true&onlyIfLeader=true&nodeName=127.0.0.1:37806_&core=collection1} status=0 QTime=1001
[junit4:junit4] 2> 17323 T1671 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49725_
[junit4:junit4] 2> 17324 T1676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9b890243000c type:delete cxid:0xf zxid:0xa7 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:49725_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:49725_
[junit4:junit4] 2> 17325 T1671 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49725_
[junit4:junit4] 2> 17329 T1762 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17329 T1700 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17329 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17329 T1693 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17329 T1729 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 17330 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 17330 T1745 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 17331 T1745 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 17332 T1700 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 17332 T1713 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 17332 T1762 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 17333 T1693 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 17340 T1763 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1364114096508/collection1
[junit4:junit4] 2> 17340 T1763 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 17341 T1763 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 17341 T1763 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 17343 T1763 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1364114096508/collection1/'
[junit4:junit4] 2> 17344 T1763 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1364114096508/collection1/lib/README' to classloader
[junit4:junit4] 2> 17345 T1763 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1364114096508/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 17380 T1763 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4] 2> 17423 T1763 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 17424 T1763 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 17428 T1763 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 17723 T1763 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 17729 T1763 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 17731 T1763 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 17747 T1763 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 17750 T1763 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 17753 T1763 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 17754 T1763 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 17755 T1763 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 17755 T1763 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 17756 T1763 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 17757 T1763 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 17757 T1763 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 17757 T1763 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1364114096508/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1364114080320/jetty4/
[junit4:junit4] 2> 17758 T1763 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7d3c85
[junit4:junit4] 2> 17758 T1763 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 17759 T1763 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1364114080320/jetty4
[junit4:junit4] 2> 17760 T1763 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1364114080320/jetty4/index/
[junit4:junit4] 2> 17760 T1763 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1364114080320/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 17760 T1763 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1364114080320/jetty4/index
[junit4:junit4] 2> 17763 T1763 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4bd1c4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@94bce9),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 17763 T1763 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 17766 T1763 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 17766 T1763 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 17767 T1763 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 17768 T1763 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 17769 T1763 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 17769 T1763 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 17769 T1763 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 17770 T1763 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 17771 T1763 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 17773 T1763 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 17775 T1763 oass.SolrIndexSearcher.<init> Opening Searcher@1a2ee47 main
[junit4:junit4] 2> 17775 T1763 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1364114080320/jetty4/tlog
[junit4:junit4] 2> 17776 T1763 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 17776 T1763 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 17780 T1764 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1a2ee47 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 17781 T1763 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 17781 T1763 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 18590 T1694 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 18592 T1694 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49725_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49725",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 18592 T1694 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 18592 T1694 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 18598 T1762 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 18599 T1745 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 18599 T1700 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 18599 T1693 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 18599 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 18599 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 18783 T1763 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 18783 T1763 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:49725 collection:collection1 shard:shard1
[junit4:junit4] 2> 18788 T1763 oasc.ZkController.register We are http://127.0.0.1:49725/collection1/ and leader is http://127.0.0.1:51388/collection1/
[junit4:junit4] 2> 18788 T1763 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49725
[junit4:junit4] 2> 18789 T1763 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 18789 T1763 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C94 name=collection1 org.apache.solr.core.SolrCore@4d14ec url=http://127.0.0.1:49725/collection1 node=127.0.0.1:49725_ C94_STATE=coll:collection1 core:collection1 props:{state=down, collection=collection1, node_name=127.0.0.1:49725_, base_url=http://127.0.0.1:49725, core=collection1}
[junit4:junit4] 2> 18789 T1765 C94 P49725 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 18791 T1765 C94 P49725 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 18791 T1763 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 18791 T1765 C94 P49725 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 18792 T1765 C94 P49725 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 18793 T1765 C94 P49725 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 18793 T1671 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 18794 T1671 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 18795 T1671 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 18801 T1671 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 18803 T1671 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 18810 T1707 oasc.CoreContainer.create Creating SolrCore 'onenodecollectioncore' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1364114083806/onenodecollectioncore
[junit4:junit4] 2> 18810 T1707 oasc.ZkController.createCollectionZkNode Check for collection zkNode:onenodecollection
[junit4:junit4] 2> 18812 T1707 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:onenodecollection
[junit4:junit4] 2> 18812 T1707 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 18814 T1707 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 18814 T1707 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection
[junit4:junit4] 2> 18818 T1707 oasc.ZkController.readConfigName Load collection config from:/collections/onenodecollection
[junit4:junit4] 2> 18820 T1707 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1364114083806/onenodecollectioncore/'
[junit4:junit4] 2> 18863 T1707 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4] 2> 18890 T1707 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 18891 T1707 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 18895 T1707 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> C93_STATE=coll:collection1 core:collection1 props:{state=recovering, collection=collection1, node_name=127.0.0.1:37806_, base_url=http://127.0.0.1:37806, shard=shard2, core=collection1}
[junit4:junit4] 2> 19195 T1748 C93 P37806 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:34938/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 19195 T1748 C93 P37806 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:37806 START replicas=[http://127.0.0.1:34938/collection1/] nUpdates=100
[junit4:junit4] 2> 19196 T1748 C93 P37806 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 19196 T1748 C93 P37806 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 19196 T1748 C93 P37806 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 19196 T1748 C93 P37806 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 19196 T1748 C93 P37806 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 19196 T1748 C93 P37806 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:34938/collection1/. core=collection1
[junit4:junit4] 2> 19196 T1748 C93 P37806 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C95 name=collection1 org.apache.solr.core.SolrCore@7a5bf2 url=http://127.0.0.1:34938/collection1 node=127.0.0.1:34938_ C95_STATE=coll:collection1 core:collection1 props:{state=active, collection=collection1, node_name=127.0.0.1:34938_, base_url=http://127.0.0.1:34938, shard=shard2, core=collection1, leader=true}
[junit4:junit4] 2> 19200 T1708 C95 P34938 REQ /get {getVersions=100&wt=javabin&version=2&qt=/get&distrib=false} status=0 QTime=1
[junit4:junit4] 2> 19200 T1709 C95 P34938 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 19202 T1709 C95 P34938 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1a527b5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@10e5d6),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 19203 T1709 C95 P34938 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 19203 T1709 C95 P34938 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1a527b5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@10e5d6),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1a527b5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@10e5d6),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> 19204 T1709 C95 P34938 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4] 2> 19204 T1709 C95 P34938 oass.SolrIndexSearcher.<init> Opening Searcher@182e183 realtime
[junit4:junit4] 2> 19204 T1709 C95 P34938 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 19204 T1709 C95 P34938 /update {softCommit=false&wt=javabin&version=2&commit=true&waitSearcher=true&openSearcher=false&commit_end_point=true} {commit=} 0 4
[junit4:junit4] 2> 19206 T1748 C93 P37806 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 19206 T1748 C93 P37806 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 19210 T1708 C95 P34938 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 19210 T1708 C95 P34938 REQ /replication {command=indexversion&wt=javabin&version=2&qt=/replication} status=0 QTime=3
[junit4:junit4] 2> 19210 T1748 C93 P37806 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 19211 T1748 C93 P37806 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 19211 T1748 C93 P37806 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 19213 T1709 C95 P34938 REQ /replication {generation=2&command=filelist&wt=javabin&version=2&qt=/replication} status=0 QTime=1
[junit4:junit4] 2> 19213 T1748 C93 P37806 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 19214 T1748 C93 P37806 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1364114080320/jetty3/index.20130324003459529
[junit4:junit4] 2> 19215 T1748 C93 P37806 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@115764d lockFactory=org.apache.lucene.store.NativeFSLockFactory@c2b709) fullCopy=false
[junit4:junit4] 2> 19217 T1706 C95 P34938 REQ /replication {generation=2&command=filecontent&checksum=true&wt=filestream&file=segments_2&qt=/replication} status=0 QTime=0
[junit4:junit4] 2> 19219 T1748 C93 P37806 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 19220 T1748 C93 P37806 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 19220 T1748 C93 P37806 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 19221 T1748 C93 P37806 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1dcf5c6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@f303f5),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1dcf5c6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@f303f5),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> 19222 T1748 C93 P37806 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4] 2> 19222 T1748 C93 P37806 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 19222 T1707 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 19222 T1748 C93 P37806 oass.SolrIndexSearcher.<init> Opening Searcher@1d4caa3 main
[junit4:junit4] 2> 19223 T1747 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1d4caa3 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 19224 T1748 C93 P37806 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 19224 T1748 C93 P37806 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 19224 T1748 C93 P37806 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 19224 T1748 C93 P37806 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 19226 T1748 C93 P37806 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 19228 T1707 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 19231 T1707 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 19247 T1707 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 19250 T1707 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 19254 T1707 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 19256 T1707 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 19256 T1707 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 19256 T1707 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 19258 T1707 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 19258 T1707 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 19259 T1707 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 19259 T1707 oasc.SolrCore.<init> [onenodecollectioncore] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1364114083806/onenodecollectioncore/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1364114080320/onenodecollection/
[junit4:junit4] 2> 19260 T1707 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7d3c85
[junit4:junit4] 2> 19260 T1707 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 19261 T1707 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1364114080320/onenodecollection
[junit4:junit4] 2> 19261 T1707 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1364114080320/onenodecollection/index/
[junit4:junit4] 2> 19262 T1707 oasc.SolrCore.initIndex WARNING [onenodecollectioncore] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1364114080320/onenodecollection/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 19262 T1707 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1364114080320/onenodecollection/index
[junit4:junit4] 2> 19264 T1707 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@ca7e3b lockFactory=org.apache.lucene.store.NativeFSLockFactory@1f09a31),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 19264 T1707 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 19266 T1707 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 19267 T1707 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 19268 T1707 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 19268 T1707 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 19269 T1707 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 19269 T1707 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 19270 T1707 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 19270 T1707 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 19271 T1707 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 19273 T1707 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 19275 T1707 oass.SolrIndexSearcher.<init> Opening Searcher@1bb7404 main
[junit4:junit4] 2> 19276 T1707 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1364114080320/onenodecollection/tlog
[junit4:junit4] 2> 19276 T1707 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 19276 T1707 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 19280 T1769 oasc.SolrCore.registerSearcher [onenodecollectioncore] Registered new searcher Searcher@1bb7404 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 19281 T1707 oasc.ZkController.publish publishing core=onenodecollectioncore state=down
[junit4:junit4] 2> 20106 T1694 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 20108 T1694 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49725_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49725",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 20114 T1694 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:37806_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:37806",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 20119 T1694 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "node_name":"127.0.0.1:34938_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34938",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"onenodecollectioncore"}
[junit4:junit4] 2> 20119 T1694 oasc.Overseer$ClusterStateUpdater.createCollection Create collection onenodecollection with numShards 1
[junit4:junit4] 2> 20119 T1694 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 20125 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 20125 T1762 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 20126 T1700 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 20125 T1693 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 20125 T1745 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 20125 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 20283 T1707 oasc.CoreContainer.registerCore registering core: onenodecollectioncore
[junit4:junit4] 2> 20284 T1707 oasc.ZkController.register Register replica - core:onenodecollectioncore address:http://127.0.0.1:34938 collection:onenodecollection shard:shard1
[junit4:junit4] 2> 20284 T1707 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection/leader_elect/shard1/election
[junit4:junit4] 2> 20300 T1676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9b8902430006 type:delete cxid:0xac zxid:0xc2 txntype:-1 reqpath:n/a Error Path:/solr/collections/onenodecollection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/onenodecollection/leaders
[junit4:junit4] 2> 20301 T1707 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 20302 T1676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9b8902430006 type:create cxid:0xad zxid:0xc3 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 20305 T1707 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 20305 T1707 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 20306 T1707 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:34938/onenodecollectioncore/
[junit4:junit4] 2> 20307 T1707 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 20307 T1707 oasc.SyncStrategy.syncToMe http://127.0.0.1:34938/onenodecollectioncore/ has no replicas
[junit4:junit4] 2> 20308 T1707 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:34938/onenodecollectioncore/
[junit4:junit4] 2> 20308 T1707 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection/leaders/shard1
[junit4:junit4] 2> 20313 T1676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9b8902430006 type:create cxid:0xb8 zxid:0xc7 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 20802 T1722 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {state=recovering&coreNodeName=5&action=PREPRECOVERY&wt=javabin&version=2&checkLive=true&onlyIfLeader=true&nodeName=127.0.0.1:49725_&core=collection1} status=0 QTime=2001
[junit4:junit4] 2> 21636 T1694 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 21647 T1762 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 21648 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 21647 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 21648 T1745 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 21647 T1700 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 21647 T1693 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 21670 T1707 oasc.ZkController.register We are http://127.0.0.1:34938/onenodecollectioncore/ and leader is http://127.0.0.1:34938/onenodecollectioncore/
[junit4:junit4] 2> 21671 T1707 oasc.ZkController.register No LogReplay needed for core=onenodecollectioncore baseURL=http://127.0.0.1:34938
[junit4:junit4] 2> 21671 T1707 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 21672 T1707 oasc.ZkController.publish publishing core=onenodecollectioncore state=active
[junit4:junit4] 2> 21674 T1707 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 21676 T1707 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {numShards=1&action=CREATE&wt=javabin&name=onenodecollectioncore&collection=onenodecollection&version=2&roles=none&dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1364114080320/onenodecollection} status=0 QTime=2866
[junit4:junit4] 2> 21677 T1671 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: onenodecollection failOnTimeout:true timeout (sec):230
[junit4:junit4] 2> 21678 T1671 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 22681 T1671 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C94_STATE=coll:collection1 core:collection1 props:{state=recovering, collection=collection1, node_name=127.0.0.1:49725_, base_url=http://127.0.0.1:49725, shard=shard1, core=collection1}
[junit4:junit4] 2> 22803 T1765 C94 P49725 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:51388/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 22803 T1765 C94 P49725 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:49725 START replicas=[http://127.0.0.1:51388/collection1/] nUpdates=100
[junit4:junit4] 2> 22804 T1765 C94 P49725 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 22804 T1765 C94 P49725 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 22805 T1765 C94 P49725 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 22805 T1765 C94 P49725 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 22805 T1765 C94 P49725 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 22806 T1765 C94 P49725 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:51388/collection1/. core=collection1
[junit4:junit4] 2> 22806 T1765 C94 P49725 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C96 name=collection1 org.apache.solr.core.SolrCore@c7daa9 url=http://127.0.0.1:51388/collection1 node=127.0.0.1:51388_ C96_STATE=coll:collection1 core:collection1 props:{state=active, collection=collection1, node_name=127.0.0.1:51388_, base_url=http://127.0.0.1:51388, shard=shard1, core=collection1, leader=true}
[junit4:junit4] 2> 22812 T1722 C96 P51388 REQ /get {getVersions=100&wt=javabin&version=2&qt=/get&distrib=false} status=0 QTime=1
[junit4:junit4] 2> 22814 T1724 C96 P51388 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 22815 T1724 C96 P51388 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6915ec lockFactory=org.apache.lucene.store.NativeFSLockFactory@1715ac8),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 22815 T1724 C96 P51388 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 22816 T1724 C96 P51388 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6915ec lockFactory=org.apache.lucene.store.NativeFSLockFactory@1715ac8),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6915ec lockFactory=org.apache.lucene.store.NativeFSLockFactory@1715ac8),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> 22816 T1724 C96 P51388 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4] 2> 22817 T1724 C96 P51388 oass.SolrIndexSearcher.<init> Opening Searcher@b702aa realtime
[junit4:junit4] 2> 22817 T1724 C96 P51388 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 22817 T1724 C96 P51388 /update {softCommit=false&wt=javabin&version=2&commit=true&waitSearcher=true&openSearcher=false&commit_end_point=true} {commit=} 0 4
[junit4:junit4] 2> 22818 T1765 C94 P49725 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 22818 T1765 C94 P49725 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 22819 T1723 C96 P51388 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 22820 T1723 C96 P51388 REQ /replication {command=indexversion&wt=javabin&version=2&qt=/replication} status=0 QTime=1
[junit4:junit4] 2> 22821 T1765 C94 P49725 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 22821 T1765 C94 P49725 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 22821 T1765 C94 P49725 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 22823 T1722 C96 P51388 REQ /replication {generation=2&command=filelist&wt=javabin&version=2&qt=/replication} status=0 QTime=0
[junit4:junit4] 2> 22823 T1765 C94 P49725 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 22825 T1765 C94 P49725 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1364114080320/jetty4/index.20130324003503140
[junit4:junit4] 2> 22825 T1765 C94 P49725 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@14e9f63 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c59326) fullCopy=false
[junit4:junit4] 2> 22829 T1724 C96 P51388 REQ /replication {generation=2&command=filecontent&checksum=true&wt=filestream&file=segments_2&qt=/replication} status=0 QTime=0
[junit4:junit4] 2> 22830 T1765 C94 P49725 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 22831 T1765 C94 P49725 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 22832 T1765 C94 P49725 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 22833 T1765 C94 P49725 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4bd1c4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@94bce9),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4bd1c4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@94bce9),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> 22834 T1765 C94 P49725 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4] 2> 22834 T1765 C94 P49725 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 22834 T1765 C94 P49725 oass.SolrIndexSearcher.<init> Opening Searcher@1fc6e46 main
[junit4:junit4] 2> 22835 T1764 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1fc6e46 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 22835 T1765 C94 P49725 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 22835 T1765 C94 P49725 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 22836 T1765 C94 P49725 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 22836 T1765 C94 P49725 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 22837 T1765 C94 P49725 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 23157 T1694 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 23158 T1694 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "node_name":"127.0.0.1:34938_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34938",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:34938__onenodecollectioncore",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"onenodecollectioncore"}
[junit4:junit4] 2> 23163 T1694 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49725_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49725",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 23170 T1745 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 23170 T1762 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 23170 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 23170 T1700 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 23170 T1693 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 23170 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 23684 T1671 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 23685 T1671 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: onenodecollection
[junit4:junit4] 2> 23685 T1671 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 23691 T1671 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:51859 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@cc1666
[junit4:junit4] 2> 23692 T1772 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:51859. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 23692 T1671 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 23693 T1772 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:51859, initiating session
[junit4:junit4] 2> 23693 T1673 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:35004
[junit4:junit4] 2> 23693 T1673 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:35004
[junit4:junit4] 2> 23694 T1675 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9b890243000d with negotiated timeout 10000 for client /127.0.0.1:35004
[junit4:junit4] 2> 23694 T1772 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:51859, sessionid = 0x13d9b890243000d, negotiated timeout = 10000
[junit4:junit4] 2> 23694 T1773 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@cc1666 name:ZooKeeperConnection Watcher:127.0.0.1:51859 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 23695 T1671 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 23727 T1676 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d9b890243000d
[junit4:junit4] 2> 23728 T1773 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 23728 T1673 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:35004 which had sessionid 0x13d9b890243000d
[junit4:junit4] 2> 23728 T1671 oaz.ZooKeeper.close Session: 0x13d9b890243000d closed
[junit4:junit4] 2> 23729 T1676 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d9b8902430004
[junit4:junit4] 2> 23729 T1700 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 23729 T1671 oaz.ZooKeeper.close Session: 0x13d9b8902430004 closed
[junit4:junit4] 2> 23729 T1673 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:34973 which had sessionid 0x13d9b8902430004
[junit4:junit4] 2> 23732 T1671 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 53297
[junit4:junit4] 2> 23732 T1671 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=11595844
[junit4:junit4] 2> 24679 T1694 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 24681 T1694 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:53297_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:53297",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 25434 T1762 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 25434 T1693 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 25434 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 25434 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 25434 T1745 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 26735 T1671 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 26735 T1671 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 26739 T1671 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@9d9b26
[junit4:junit4] 2> 26745 T1671 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 26746 T1671 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 26747 T1671 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 26747 T1671 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 26748 T1671 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 26749 T1671 oasc.CachingDirectoryFactory.close Closing directory when closing factory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1364114080320/control/data/index
[junit4:junit4] 2> 26750 T1671 oasc.CachingDirectoryFactory.closeDirectory Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1364114080320/control/data/index
[junit4:junit4] 2> 26751 T1671 oasc.CachingDirectoryFactory.close Closing directory when closing factory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1364114080320/control/data
[junit4:junit4] 2> 26751 T1671 oasc.CachingDirectoryFactory.closeDirectory Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1364114080320/control/data
[junit4:junit4] 2> 26753 T1694 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89398580368048131-127.0.0.1:53297_-n_0000000000) am no longer a leader.
[junit4:junit4] 2> 26753 T1676 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d9b8902430003
[junit4:junit4] 2> 26754 T1693 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 26754 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 26755 T1673 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:34972 which had sessionid 0x13d9b8902430003
[junit4:junit4] 2> 26755 T1762 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 26755 T1671 oaz.ZooKeeper.close Session: 0x13d9b8902430003 closed
[junit4:junit4] 2> 26756 T1713 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 26755 T1745 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 26755 T1762 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 26756 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 26757 T1729 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 26756 T1745 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 26759 T1676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9b8902430006 type:delete cxid:0xc9 zxid:0xe3 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4] 2> 26760 T1713 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 26763 T1713 oasc.Overseer.start Overseer (id=89398580368048134-127.0.0.1:34938_-n_0000000001) starting
[junit4:junit4] 2> 26764 T1676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9b8902430006 type:create cxid:0xce zxid:0xe5 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 26765 T1676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9b8902430006 type:create cxid:0xcf zxid:0xe6 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 26769 T1676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9b8902430006 type:create cxid:0xd1 zxid:0xe7 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 26771 T1676 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9b8902430006 type:create cxid:0xd3 zxid:0xe8 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 26773 T1775 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 26775 T1774 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 26778 T1774 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 26780 T1774 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:53297_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:53297",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 26780 T1671 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
[junit4:junit4] 2> 26785 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 26785 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 26785 T1762 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 26785 T1745 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 26832 T1671 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 34938
[junit4:junit4] 2> 26833 T1671 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=20209315
[junit4:junit4] 2> 28055 T1693 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 28055 T1693 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 28055 T1693 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 28293 T1774 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 28294 T1774 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:34938_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34938",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "core":"collection1"}
[junit4:junit4] 2> 28300 T1774 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "node_name":"127.0.0.1:34938_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34938",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:34938__onenodecollectioncore",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"onenodecollectioncore"}
[junit4:junit4] 2> 28305 T1713 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 28305 T1729 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 28305 T1762 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 28305 T1745 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 29836 T1671 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 29837 T1671 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 29839 T1671 oasc.ZkController.publish publishing core=onenodecollectioncore state=down
[junit4:junit4] 2> 29843 T1671 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@7a5bf2
[junit4:junit4] 2> 29848 T1671 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 29849 T1671 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 29849 T1671 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 29850 T1671 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 29854 T1671 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 29855 T1671 oasc.CachingDirectoryFactory.close Closing directory when closing factory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1364114080320/jetty1/index
[junit4:junit4] 2> 29855 T1671 oasc.CachingDirectoryFactory.closeDirectory Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1364114080320/jetty1/index
[junit4:junit4] 2> 29856 T1671 oasc.CachingDirectoryFactory.close Closing directory when closing factory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1364114080320/jetty1
[junit4:junit4] 2> 29857 T1671 oasc.CachingDirectoryFactory.closeDirectory Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1364114080320/jetty1
[junit4:junit4] 2> 29857 T1671 oasc.SolrCore.close [onenodecollectioncore] CLOSING SolrCore org.apache.solr.core.SolrCore@2aff27
[junit4:junit4] 2> 29865 T1671 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 29865 T1671 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 29866 T1671 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 29867 T1671 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 29868 T1671 oasc.SolrCore.closeSearcher [onenodecollectioncore] Closing main searcher on request.
[junit4:junit4] 2> 29869 T1671 oasc.CachingDirectoryFactory.close Closing directory when closing factory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1364114080320/onenodecollection
[junit4:junit4] 2> 29869 T1671 oasc.CachingDirectoryFactory.closeDirectory Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1364114080320/onenodecollection
[junit4:junit4] 2> 29870 T1671 oasc.CachingDirectoryFactory.close Closing directory when closing factory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1364114080320/onenodecollection/index
[junit4:junit4] 2> 29871 T1671 oasc.CachingDirectoryFactory.closeDirectory Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZk2Test-1364114080320/onenodecollection/index
[junit4:junit4] 2> 29872 T1774 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89398580368048134-127.0.0.1:34938_-n_0000000001) am no longer a leader.
[junit4:junit4] 2> 29872 T1676 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid:
[...truncated too long message...]
4:junit4] 1> "router":"compositeId"}}
[junit4:junit4] 1> /solr/live_nodes (5)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:52407_n_%2Fc (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:45164_n_%2Fc (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:35792_n_%2Fc (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:37707_n_%2Fc (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:41759_n_%2Fc (0)
[junit4:junit4] 1> /solr/overseer (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1> /solr/overseer/queue-work (0)
[junit4:junit4] 1> /solr/overseer/collection-queue-work (0)
[junit4:junit4] 1> /solr/overseer/queue (0)
[junit4:junit4] 1>
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=UnloadDistributedZkTest -Dtests.method=testDistribSearch -Dtests.seed=9753738F1061972F -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=sl_SI -Dtests.timezone=America/Sitka -Dtests.file.encoding=ISO-8859-1
[junit4:junit4] FAILURE 141s J1 | UnloadDistributedZkTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: Still found shard2 in collection test_unload_shard_and_collection
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([9753738F1061972F:16B5FD97673EF713]:0)
[junit4:junit4] > at org.junit.Assert.fail(Assert.java:93)
[junit4:junit4] > at org.apache.solr.cloud.UnloadDistributedZkTest.testUnloadShardAndCollection(UnloadDistributedZkTest.java:127)
[junit4:junit4] > at org.apache.solr.cloud.UnloadDistributedZkTest.doTest(UnloadDistributedZkTest.java:77)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
[junit4:junit4] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4:junit4] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[junit4:junit4] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4:junit4] > at java.lang.reflect.Method.invoke(Method.java:487)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4] > at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4:junit4] > at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4:junit4] > at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4] > at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
[junit4:junit4] > at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4] > at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4] > at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4] > at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4] > at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4] > at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4] > at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2> 140665 T1862 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 140674 T1861 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 2 leaked thread(s).
[junit4:junit4] 2> 141493 T1865 oazs.SessionTrackerImpl.run SessionTrackerImpl exited loop!
[junit4:junit4] 2> NOTE: test params are: codec=Lucene3x, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=sl_SI, timezone=America/Sitka
[junit4:junit4] 2> NOTE: Linux 3.2.0-39-generic i386/Oracle Corporation 1.8.0-ea (32-bit)/cpus=8,threads=1,free=187895080,total=320864256
[junit4:junit4] 2> NOTE: All tests run in this JVM: [SpellCheckCollatorTest, SimplePostToolTest, RequestHandlersTest, IndexSchemaRuntimeFieldTest, SortByFunctionTest, TestLazyCores, SolrRequestParserTest, TestIndexingPerformance, TestAtomicUpdateErrorCases, SuggesterWFSTTest, TestRangeQuery, TestAnalyzedSuggestions, TestNumberUtils, TestReload, TestDefaultSimilarityFactory, XsltUpdateRequestHandlerTest, TestElisionMultitermQuery, ZkSolrClientTest, TestUtils, TestRTGBase, NumericFieldsTest, DebugComponentTest, TimeZoneUtilsTest, TestPostingsSolrHighlighter, URLClassifyProcessorTest, HardAutoCommitTest, TestSolrDiscoveryProperties, TestJmxIntegration, UniqFieldsUpdateProcessorFactoryTest, StandardRequestHandlerTest, TestDistributedSearch, XmlUpdateRequestHandlerTest, UpdateParamsTest, BadIndexSchemaTest, FileUtilsTest, EchoParamsTest, TestShardHandlerFactory, IndexBasedSpellCheckerTest, TestCollationField, TestUpdate, ConvertedLegacyTest, TestSolrXMLSerializer, SpellCheckComponentTest, NotRequiredUniqueKeyTest, TestFieldCollectionResource, ClusterStateTest, SampleTest, TestLuceneMatchVersion, TestSolr4Spatial, TestCSVLoader, SolrCmdDistributorTest, FieldAnalysisRequestHandlerTest, SolrIndexConfigTest, TestLMJelinekMercerSimilarityFactory, OutputWriterTest, TestCopyFieldCollectionResource, TestBinaryField, CopyFieldTest, TestMergePolicyConfig, HighlighterConfigTest, TestMultiCoreConfBootstrap, OpenCloseCoreStressTest, SuggesterTSTTest, TestPropInject, ShardRoutingCustomTest, NoCacheHeaderTest, ChaosMonkeySafeLeaderTest, UnloadDistributedZkTest]
[junit4:junit4] Completed on J1 in 141.71s, 1 test, 1 failure <<< FAILURES!
[...truncated 412 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:381: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:361: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:183: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:449: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1212: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:876: There were test failures: 273 suites, 1149 tests, 1 error, 1 failure, 13 ignored (7 assumptions)
Total time: 44 minutes 5 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Description set: Java: 32bit/jdk1.8.0-ea-b82 -server -XX:+UseG1GC -XX:MarkStackSize=256K
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b82) - Build #
4806 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/4806/
Java: 32bit/jdk1.8.0-ea-b82 -server -XX:+UseConcMarkSweepGC
2 tests failed.
REGRESSION: org.apache.solr.cloud.UnloadDistributedZkTest.testDistribSearch
Error Message:
Still found shard2 in collection test_unload_shard_and_collection
Stack Trace:
java.lang.AssertionError: Still found shard2 in collection test_unload_shard_and_collection
at __randomizedtesting.SeedInfo.seed([12AF573644669E6B:9349D92E3339FE57]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.UnloadDistributedZkTest.testUnloadShardAndCollection(UnloadDistributedZkTest.java:127)
at org.apache.solr.cloud.UnloadDistributedZkTest.doTest(UnloadDistributedZkTest.java:77)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:487)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
FAILED: org.apache.solr.cloud.BasicDistributedZk2Test.testDistribSearch
Error Message:
Server at http://127.0.0.1:34480/onenodecollectioncore returned non ok status:404, message:Can not find: /onenodecollectioncore/update
Stack Trace:
org.apache.solr.common.SolrException: Server at http://127.0.0.1:34480/onenodecollectioncore returned non ok status:404, message:Can not find: /onenodecollectioncore/update
at __randomizedtesting.SeedInfo.seed([12AF573644669E6B:9349D92E3339FE57]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:375)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181)
at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:116)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:102)
at org.apache.solr.cloud.BasicDistributedZk2Test.testNodeWithoutCollectionForwarding(BasicDistributedZk2Test.java:197)
at org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:89)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:487)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Build Log:
[...truncated 9003 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.UnloadDistributedZkTest
[junit4:junit4] 2> 0 T31 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4] 2> 9 T31 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1364129512471
[junit4:junit4] 2> 13 T31 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 16 T32 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 42 T32 oaz.Environment.logEnv Server environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT
[junit4:junit4] 2> 42 T32 oaz.Environment.logEnv Server environment:host.name=serv1
[junit4:junit4] 2> 43 T32 oaz.Environment.logEnv Server environment:java.version=1.8.0-ea
[junit4:junit4] 2> 43 T32 oaz.Environment.logEnv Server environment:java.vendor=Oracle Corporation
[junit4:junit4] 2> 43 T32 oaz.Environment.logEnv Server environment:java.home=/var/lib/jenkins/tools/java/32bit/jdk1.8.0-ea-b82/jre
[junit4:junit4] 2> 43 T32 oaz.Environment.logEnv Server environment:java.class.path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/classes/test:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-test-framework/classes/java:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/test-framework/classes/java:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/codecs/classes/java:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/classes/java:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/classes/java:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/analysis/common/lucene-analyzers-common-4.3-SNAPSHOT.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-4.3-SNAPSHOT.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-4.3-SNAPSHOT.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/codecs/lucene-codecs-4.3-SNAPSHOT.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/highlighter/lucene-highlighter-4.3-SNAPSHOT.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/memory/lucene-memory-4.3-SNAPSHOT.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/misc/lucene-misc-4.3-SNAPSHOT.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/spatial/lucene-spatial-4.3-SNAPSHOT.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/suggest/lucene-suggest-4.3-SNAPSHOT.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/grouping/lucene-grouping-4.3-SNAPSHOT.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/queries/lucene-queries-4.3-SNAPSHOT.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/queryparser/lucene-queryparser-4.3-SNAPSHOT.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/lib/cglib-nodep-2.2.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/lib/commons-cli-1.2.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/lib/commons-codec-1.7.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/lib/commons-fileupload-1.2.1.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/lib/commons-lang-2.6.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/lib/easymock-3.0.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/lib/guava-13.0.1.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/lib/javax.servlet-api-3.0.1.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/lib/objenesis-1.2.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/lib/org.restlet-2.1.1.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/lib/org.restlet.ext.servlet-2.1.1.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/lib/spatial4j-0.3.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/lib/commons-io-2.1.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/lib/httpclient-4.2.3.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/lib/httpcore-4.2.2.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/lib/httpmime-4.2.3.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/lib/jcl-over-slf4j-1.6.4.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/lib/noggit-0.5.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/lib/slf4j-api-1.6.4.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/lib/slf4j-jdk14-1.6.4.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/lib/wstx-asl-3.2.7.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/lib/zookeeper-3.4.5.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/example/lib/jetty-continuation-8.1.8.v20121106.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/example/lib/jetty-deploy-8.1.8.v20121106.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/example/lib/jetty-http-8.1.8.v20121106.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/example/lib/jetty-io-8.1.8.v20121106.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/example/lib/jetty-jmx-8.1.8.v20121106.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/example/lib/jetty-security-8.1.8.v20121106.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/example/lib/jetty-server-8.1.8.v20121106.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/example/lib/jetty-servlet-8.1.8.v20121106.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/example/lib/jetty-util-8.1.8.v20121106.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/example/lib/jetty-webapp-8.1.8.v20121106.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/example/lib/jetty-xml-8.1.8.v20121106.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/example/lib/servlet-api-3.0.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/example/example-DIH/solr/db/lib/derby-10.9.1.0.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/core/classes/java:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/test-framework/lib/junit-4.10.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/test-framework/lib/randomizedtesting-runner-2.0.8.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-launcher.jar:/var/lib/jenkins/.ant/lib/ivy-2.3.0.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jai.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-swing.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-oro.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jmf.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-xalan2.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-javamail.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-resolver.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-testutil.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-commons-logging.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-log4j.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-junit.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jsch.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-commons-net.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bsf.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jdepend.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-netrexx.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-regexp.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-junit4.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bcel.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-antlr.jar:/var/lib/jenkins/tools/java/32bit/jdk1.8.0-ea-b82/lib/tools.jar:/var/lib/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.0.8.jar
[junit4:junit4] 2> 44 T32 oaz.Environment.logEnv Server environment:java.library.path=/usr/java/packages/lib/i386:/lib:/usr/lib
[junit4:junit4] 2> 44 T32 oaz.Environment.logEnv Server environment:java.io.tmpdir=.
[junit4:junit4] 2> 45 T32 oaz.Environment.logEnv Server environment:java.compiler=<NA>
[junit4:junit4] 2> 45 T32 oaz.Environment.logEnv Server environment:os.name=Linux
[junit4:junit4] 2> 45 T32 oaz.Environment.logEnv Server environment:os.arch=i386
[junit4:junit4] 2> 45 T32 oaz.Environment.logEnv Server environment:os.version=3.2.0-39-generic
[junit4:junit4] 2> 46 T32 oaz.Environment.logEnv Server environment:user.name=jenkins
[junit4:junit4] 2> 46 T32 oaz.Environment.logEnv Server environment:user.home=/var/lib/jenkins
[junit4:junit4] 2> 46 T32 oaz.Environment.logEnv Server environment:user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 60 T32 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4] 2> 60 T32 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4] 2> 60 T32 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4] 2> 62 T32 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4] 2> 88 T32 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-1364129512470/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4] 2> 115 T31 oasc.ZkTestServer.run start zk server on port:38828
[junit4:junit4] 2> 139 T31 oaz.Environment.logEnv Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT
[junit4:junit4] 2> 140 T31 oaz.Environment.logEnv Client environment:host.name=serv1
[junit4:junit4] 2> 141 T31 oaz.Environment.logEnv Client environment:java.version=1.8.0-ea
[junit4:junit4] 2> 141 T31 oaz.Environment.logEnv Client environment:java.vendor=Oracle Corporation
[junit4:junit4] 2> 142 T31 oaz.Environment.logEnv Client environment:java.home=/var/lib/jenkins/tools/java/32bit/jdk1.8.0-ea-b82/jre
[junit4:junit4] 2> 142 T31 oaz.Environment.logEnv Client environment:java.class.path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/classes/test:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-test-framework/classes/java:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/test-framework/classes/java:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/codecs/classes/java:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/classes/java:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/classes/java:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/analysis/common/lucene-analyzers-common-4.3-SNAPSHOT.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-4.3-SNAPSHOT.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-4.3-SNAPSHOT.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/codecs/lucene-codecs-4.3-SNAPSHOT.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/highlighter/lucene-highlighter-4.3-SNAPSHOT.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/memory/lucene-memory-4.3-SNAPSHOT.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/misc/lucene-misc-4.3-SNAPSHOT.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/spatial/lucene-spatial-4.3-SNAPSHOT.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/suggest/lucene-suggest-4.3-SNAPSHOT.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/grouping/lucene-grouping-4.3-SNAPSHOT.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/queries/lucene-queries-4.3-SNAPSHOT.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/queryparser/lucene-queryparser-4.3-SNAPSHOT.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/lib/cglib-nodep-2.2.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/lib/commons-cli-1.2.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/lib/commons-codec-1.7.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/lib/commons-fileupload-1.2.1.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/lib/commons-lang-2.6.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/lib/easymock-3.0.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/lib/guava-13.0.1.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/lib/javax.servlet-api-3.0.1.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/lib/objenesis-1.2.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/lib/org.restlet-2.1.1.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/lib/org.restlet.ext.servlet-2.1.1.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/core/lib/spatial4j-0.3.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/lib/commons-io-2.1.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/lib/httpclient-4.2.3.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/lib/httpcore-4.2.2.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/lib/httpmime-4.2.3.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/lib/jcl-over-slf4j-1.6.4.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/lib/noggit-0.5.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/lib/slf4j-api-1.6.4.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/lib/slf4j-jdk14-1.6.4.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/lib/wstx-asl-3.2.7.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/lib/zookeeper-3.4.5.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/example/lib/jetty-continuation-8.1.8.v20121106.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/example/lib/jetty-deploy-8.1.8.v20121106.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/example/lib/jetty-http-8.1.8.v20121106.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/example/lib/jetty-io-8.1.8.v20121106.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/example/lib/jetty-jmx-8.1.8.v20121106.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/example/lib/jetty-security-8.1.8.v20121106.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/example/lib/jetty-server-8.1.8.v20121106.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/example/lib/jetty-servlet-8.1.8.v20121106.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/example/lib/jetty-util-8.1.8.v20121106.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/example/lib/jetty-webapp-8.1.8.v20121106.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/example/lib/jetty-xml-8.1.8.v20121106.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/example/lib/servlet-api-3.0.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/example/example-DIH/solr/db/lib/derby-10.9.1.0.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/core/classes/java:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/test-framework/lib/junit-4.10.jar:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/test-framework/lib/randomizedtesting-runner-2.0.8.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-launcher.jar:/var/lib/jenkins/.ant/lib/ivy-2.3.0.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jai.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-swing.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-oro.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jmf.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-xalan2.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-javamail.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-resolver.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-testutil.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-commons-logging.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-log4j.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-junit.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jsch.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-commons-net.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bsf.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jdepend.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-netrexx.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-regexp.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-junit4.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bcel.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-antlr.jar:/var/lib/jenkins/tools/java/32bit/jdk1.8.0-ea-b82/lib/tools.jar:/var/lib/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.0.8.jar
[junit4:junit4] 2> 144 T31 oaz.Environment.logEnv Client environment:java.library.path=/usr/java/packages/lib/i386:/lib:/usr/lib
[junit4:junit4] 2> 145 T31 oaz.Environment.logEnv Client environment:java.io.tmpdir=.
[junit4:junit4] 2> 145 T31 oaz.Environment.logEnv Client environment:java.compiler=<NA>
[junit4:junit4] 2> 146 T31 oaz.Environment.logEnv Client environment:os.name=Linux
[junit4:junit4] 2> 147 T31 oaz.Environment.logEnv Client environment:os.arch=i386
[junit4:junit4] 2> 147 T31 oaz.Environment.logEnv Client environment:os.version=3.2.0-39-generic
[junit4:junit4] 2> 148 T31 oaz.Environment.logEnv Client environment:user.name=jenkins
[junit4:junit4] 2> 148 T31 oaz.Environment.logEnv Client environment:user.home=/var/lib/jenkins
[junit4:junit4] 2> 149 T31 oaz.Environment.logEnv Client environment:user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 152 T31 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:38828 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@1d0efc6
[junit4:junit4] 2> 178 T31 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 182 T37 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:38828. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 183 T37 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:38828, initiating session
[junit4:junit4] 2> 183 T33 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:40090
[junit4:junit4] 2> 192 T33 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:40090
[junit4:junit4] 2> 197 T35 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4] 2> 208 T35 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9c747c6f0000 with negotiated timeout 10000 for client /127.0.0.1:40090
[junit4:junit4] 2> 208 T37 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:38828, sessionid = 0x13d9c747c6f0000, negotiated timeout = 10000
[junit4:junit4] 2> 211 T38 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1d0efc6 name:ZooKeeperConnection Watcher:127.0.0.1:38828 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 211 T31 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 213 T31 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 236 T36 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d9c747c6f0000
[junit4:junit4] 2> 237 T38 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 237 T31 oaz.ZooKeeper.close Session: 0x13d9c747c6f0000 closed
[junit4:junit4] 2> 238 T33 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:40090 which had sessionid 0x13d9c747c6f0000
[junit4:junit4] 2> 238 T31 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:38828/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@7c0acf
[junit4:junit4] 2> 239 T31 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 239 T39 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:38828. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 240 T39 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:38828, initiating session
[junit4:junit4] 2> 240 T33 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:40091
[junit4:junit4] 2> 241 T33 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:40091
[junit4:junit4] 2> 242 T35 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9c747c6f0001 with negotiated timeout 10000 for client /127.0.0.1:40091
[junit4:junit4] 2> 242 T39 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:38828, sessionid = 0x13d9c747c6f0001, negotiated timeout = 10000
[junit4:junit4] 2> 242 T40 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7c0acf name:ZooKeeperConnection Watcher:127.0.0.1:38828/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 243 T31 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 248 T31 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 253 T31 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 259 T31 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 263 T31 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 268 T31 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 276 T31 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 284 T31 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 284 T31 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 290 T31 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 290 T31 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 297 T31 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 298 T31 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 304 T31 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 305 T31 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 310 T31 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 311 T31 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 315 T31 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 316 T31 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 321 T31 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 322 T31 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 327 T31 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 327 T31 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 332 T36 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d9c747c6f0001
[junit4:junit4] 2> 333 T40 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 333 T31 oaz.ZooKeeper.close Session: 0x13d9c747c6f0001 closed
[junit4:junit4] 2> 333 T33 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:40091 which had sessionid 0x13d9c747c6f0001
[junit4:junit4] 2> 412 T31 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 418 T31 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:48031
[junit4:junit4] 2> 419 T31 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 419 T31 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 420 T31 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-controljetty-1364129512795
[junit4:junit4] 2> 420 T31 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-controljetty-1364129512795/solr.xml
[junit4:junit4] 2> 421 T31 oasc.CoreContainer.<init> New CoreContainer 6066463
[junit4:junit4] 2> 421 T31 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-controljetty-1364129512795/'
[junit4:junit4] 2> 422 T31 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-controljetty-1364129512795/'
[junit4:junit4] 2> 443 T31 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 443 T31 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 444 T31 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 444 T31 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 445 T31 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 445 T31 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 445 T31 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 446 T31 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 446 T31 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 447 T31 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 537 T31 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 555 T31 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:38828/solr
[junit4:junit4] 2> 563 T31 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 564 T31 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:38828 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@67b42d
[junit4:junit4] 2> 565 T31 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 565 T50 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:38828. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 566 T50 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:38828, initiating session
[junit4:junit4] 2> 566 T33 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:40092
[junit4:junit4] 2> 567 T33 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:40092
[junit4:junit4] 2> 569 T35 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9c747c6f0002 with negotiated timeout 20000 for client /127.0.0.1:40092
[junit4:junit4] 2> 569 T50 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:38828, sessionid = 0x13d9c747c6f0002, negotiated timeout = 20000
[junit4:junit4] 2> 570 T51 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@67b42d name:ZooKeeperConnection Watcher:127.0.0.1:38828 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 570 T31 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 572 T36 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d9c747c6f0002
[junit4:junit4] 2> 573 T51 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 573 T31 oaz.ZooKeeper.close Session: 0x13d9c747c6f0002 closed
[junit4:junit4] 2> 573 T33 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:40092 which had sessionid 0x13d9c747c6f0002
[junit4:junit4] 2> 575 T31 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 580 T31 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:38828/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@f8f755
[junit4:junit4] 2> 581 T31 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 581 T52 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:38828. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 582 T52 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:38828, initiating session
[junit4:junit4] 2> 582 T33 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:40093
[junit4:junit4] 2> 582 T33 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:40093
[junit4:junit4] 2> 583 T35 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9c747c6f0003 with negotiated timeout 20000 for client /127.0.0.1:40093
[junit4:junit4] 2> 584 T52 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:38828, sessionid = 0x13d9c747c6f0003, negotiated timeout = 20000
[junit4:junit4] 2> 584 T53 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f8f755 name:ZooKeeperConnection Watcher:127.0.0.1:38828/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 584 T31 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 590 T31 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 596 T36 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c747c6f0003 type:create cxid:0x7 zxid:0x1b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 598 T31 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 605 T31 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 608 T31 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48031_
[junit4:junit4] 2> 612 T36 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c747c6f0003 type:delete cxid:0x12 zxid:0x1e txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:48031_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:48031_
[junit4:junit4] 2> 618 T31 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:48031_
[junit4:junit4] 2> 624 T31 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 637 T36 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c747c6f0003 type:delete cxid:0x20 zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4] 2> 638 T31 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 643 T31 oasc.Overseer.start Overseer (id=89399591735066627-127.0.0.1:48031_-n_0000000000) starting
[junit4:junit4] 2> 644 T36 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c747c6f0003 type:create cxid:0x25 zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 651 T36 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c747c6f0003 type:create cxid:0x26 zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 653 T36 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c747c6f0003 type:create cxid:0x28 zxid:0x27 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 655 T31 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 663 T36 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c747c6f0003 type:create cxid:0x2e zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 665 T55 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 665 T31 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 670 T31 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 673 T31 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 681 T54 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 683 T31 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 684 T31 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 699 T31 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 709 T31 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 711 T31 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:38828/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@1469ad5
[junit4:junit4] 2> 712 T31 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 712 T57 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:38828. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 713 T57 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:38828, initiating session
[junit4:junit4] 2> 713 T33 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:40094
[junit4:junit4] 2> 713 T33 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:40094
[junit4:junit4] 2> 714 T35 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9c747c6f0004 with negotiated timeout 10000 for client /127.0.0.1:40094
[junit4:junit4] 2> 714 T57 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:38828, sessionid = 0x13d9c747c6f0004, negotiated timeout = 10000
[junit4:junit4] 2> 715 T58 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1469ad5 name:ZooKeeperConnection Watcher:127.0.0.1:38828/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 715 T31 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 717 T31 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 722 T31 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 794 T31 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 796 T31 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:43934
[junit4:junit4] 2> 797 T31 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 798 T31 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 798 T31 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty1-1364129513183
[junit4:junit4] 2> 798 T31 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty1-1364129513183/solr.xml
[junit4:junit4] 2> 799 T31 oasc.CoreContainer.<init> New CoreContainer 15549231
[junit4:junit4] 2> 799 T31 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty1-1364129513183/'
[junit4:junit4] 2> 800 T31 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty1-1364129513183/'
[junit4:junit4] 2> 821 T31 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 822 T31 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 823 T31 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 823 T31 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 824 T31 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 825 T31 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 825 T31 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 826 T31 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 826 T31 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 827 T31 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 834 T31 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 847 T31 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:38828/solr
[junit4:junit4] 2> 847 T31 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 848 T31 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:38828 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@8db5ca
[junit4:junit4] 2> 849 T31 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 849 T68 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:38828. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 850 T68 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:38828, initiating session
[junit4:junit4] 2> 851 T33 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:40095
[junit4:junit4] 2> 851 T33 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:40095
[junit4:junit4] 2> 852 T35 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9c747c6f0005 with negotiated timeout 20000 for client /127.0.0.1:40095
[junit4:junit4] 2> 853 T68 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:38828, sessionid = 0x13d9c747c6f0005, negotiated timeout = 20000
[junit4:junit4] 2> 853 T69 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@8db5ca name:ZooKeeperConnection Watcher:127.0.0.1:38828 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 853 T31 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 855 T36 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d9c747c6f0005
[junit4:junit4] 2> 856 T33 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:40095 which had sessionid 0x13d9c747c6f0005
[junit4:junit4] 2> 856 T31 oaz.ZooKeeper.close Session: 0x13d9c747c6f0005 closed
[junit4:junit4] 2> 856 T31 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 860 T31 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:38828/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@188e830
[junit4:junit4] 2> 861 T69 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 861 T31 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 864 T70 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:38828. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 865 T70 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:38828, initiating session
[junit4:junit4] 2> 865 T33 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:40096
[junit4:junit4] 2> 866 T33 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:40096
[junit4:junit4] 2> 867 T35 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9c747c6f0006 with negotiated timeout 20000 for client /127.0.0.1:40096
[junit4:junit4] 2> 867 T70 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:38828, sessionid = 0x13d9c747c6f0006, negotiated timeout = 20000
[junit4:junit4] 2> 868 T71 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@188e830 name:ZooKeeperConnection Watcher:127.0.0.1:38828/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 868 T31 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 869 T36 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c747c6f0006 type:create cxid:0x1 zxid:0x30 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 871 T36 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c747c6f0006 type:create cxid:0x3 zxid:0x31 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 875 T31 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1879 T31 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43934_
[junit4:junit4] 2> 1881 T36 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c747c6f0006 type:delete cxid:0xd zxid:0x32 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:43934_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:43934_
[junit4:junit4] 2> 1883 T31 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:43934_
[junit4:junit4] 2> 1894 T53 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1894 T58 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1894 T71 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1908 T31 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1909 T31 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1910 T31 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1976 T31 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 1978 T31 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:45516
[junit4:junit4] 2> 1979 T31 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1980 T31 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1980 T31 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty2-1364129514377
[junit4:junit4] 2> 1981 T31 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty2-1364129514377/solr.xml
[junit4:junit4] 2> 1981 T31 oasc.CoreContainer.<init> New CoreContainer 32428294
[junit4:junit4] 2> 1981 T31 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty2-1364129514377/'
[junit4:junit4] 2> 1982 T31 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty2-1364129514377/'
[junit4:junit4] 2> 2002 T31 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 2002 T31 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 2003 T31 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 2003 T31 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 2004 T31 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 2004 T31 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 2004 T31 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 2005 T31 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 2005 T31 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 2006 T31 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 2012 T31 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 2027 T31 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:38828/solr
[junit4:junit4] 2> 2027 T31 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 2028 T31 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:38828 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@91a0ba
[junit4:junit4] 2> 2029 T31 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2029 T82 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:38828. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 2030 T82 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:38828, initiating session
[junit4:junit4] 2> 2031 T33 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:40099
[junit4:junit4] 2> 2032 T33 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:40099
[junit4:junit4] 2> 2033 T35 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9c747c6f0007 with negotiated timeout 20000 for client /127.0.0.1:40099
[junit4:junit4] 2> 2033 T82 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:38828, sessionid = 0x13d9c747c6f0007, negotiated timeout = 20000
[junit4:junit4] 2> 2034 T83 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@91a0ba name:ZooKeeperConnection Watcher:127.0.0.1:38828 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2034 T31 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2037 T36 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d9c747c6f0007
[junit4:junit4] 2> 2038 T83 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 2039 T33 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:40099 which had sessionid 0x13d9c747c6f0007
[junit4:junit4] 2> 2038 T31 oaz.ZooKeeper.close Session: 0x13d9c747c6f0007 closed
[junit4:junit4] 2> 2039 T31 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 2043 T31 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:38828/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@1f49823
[junit4:junit4] 2> 2044 T31 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2044 T84 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:38828. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 2044 T84 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:38828, initiating session
[junit4:junit4] 2> 2045 T33 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:40100
[junit4:junit4] 2> 2046 T33 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:40100
[junit4:junit4] 2> 2047 T35 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9c747c6f0008 with negotiated timeout 20000 for client /127.0.0.1:40100
[junit4:junit4] 2> 2047 T84 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:38828, sessionid = 0x13d9c747c6f0008, negotiated timeout = 20000
[junit4:junit4] 2> 2048 T85 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f49823 name:ZooKeeperConnection Watcher:127.0.0.1:38828/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2048 T31 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2050 T36 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c747c6f0008 type:create cxid:0x1 zxid:0x38 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 2053 T36 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c747c6f0008 type:create cxid:0x3 zxid:0x39 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 2058 T31 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 3066 T31 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45516_
[junit4:junit4] 2> 3068 T36 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c747c6f0008 type:delete cxid:0xd zxid:0x3a txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:45516_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:45516_
[junit4:junit4] 2> 3069 T31 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:45516_
[junit4:junit4] 2> 3077 T71 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 3077 T58 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 3077 T53 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 3078 T85 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 3087 T31 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 3088 T31 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 3089 T31 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 3156 T31 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 3159 T31 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:44754
[junit4:junit4] 2> 3160 T31 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 3160 T31 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 3161 T31 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty3-1364129515557
[junit4:junit4] 2> 3161 T31 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty3-1364129515557/solr.xml
[junit4:junit4] 2> 3162 T31 oasc.CoreContainer.<init> New CoreContainer 10694544
[junit4:junit4] 2> 3162 T31 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty3-1364129515557/'
[junit4:junit4] 2> 3163 T31 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty3-1364129515557/'
[junit4:junit4] 2> 3183 T31 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 3184 T31 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 3184 T31 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 3184 T31 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 3185 T31 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 3185 T31 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 3185 T31 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 3186 T31 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 3186 T31 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 3187 T31 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 3194 T31 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 3208 T31 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:38828/solr
[junit4:junit4] 2> 3209 T31 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 3209 T31 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:38828 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@25afb5
[junit4:junit4] 2> 3210 T31 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 3211 T96 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:38828. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 3212 T96 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:38828, initiating session
[junit4:junit4] 2> 3212 T33 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:40101
[junit4:junit4] 2> 3213 T33 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:40101
[junit4:junit4] 2> 3215 T35 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9c747c6f0009 with negotiated timeout 20000 for client /127.0.0.1:40101
[junit4:junit4] 2> 3215 T96 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:38828, sessionid = 0x13d9c747c6f0009, negotiated timeout = 20000
[junit4:junit4] 2> 3216 T97 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@25afb5 name:ZooKeeperConnection Watcher:127.0.0.1:38828 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 3216 T31 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 3219 T36 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d9c747c6f0009
[junit4:junit4] 2> 3220 T97 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 3220 T31 oaz.ZooKeeper.close Session: 0x13d9c747c6f0009 closed
[junit4:junit4] 2> 3220 T33 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:40101 which had sessionid 0x13d9c747c6f0009
[junit4:junit4] 2> 3221 T31 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 3224 T31 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:38828/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@1019e13
[junit4:junit4] 2> 3225 T31 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 3225 T98 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:38828. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 3226 T98 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:38828, initiating session
[junit4:junit4] 2> 3226 T33 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:40102
[junit4:junit4] 2> 3227 T33 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:40102
[junit4:junit4] 2> 3229 T35 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9c747c6f000a with negotiated timeout 20000 for client /127.0.0.1:40102
[junit4:junit4] 2> 3229 T98 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:38828, sessionid = 0x13d9c747c6f000a, negotiated timeout = 20000
[junit4:junit4] 2> 3230 T99 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1019e13 name:ZooKeeperConnection Watcher:127.0.0.1:38828/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 3231 T31 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 3232 T36 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c747c6f000a type:create cxid:0x1 zxid:0x40 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 3236 T36 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c747c6f000a type:create cxid:0x3 zxid:0x41 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 3241 T31 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 4247 T31 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44754_
[junit4:junit4] 2> 4248 T36 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c747c6f000a type:delete cxid:0xd zxid:0x42 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:44754_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:44754_
[junit4:junit4] 2> 4250 T31 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:44754_
[junit4:junit4] 2> 4254 T53 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 4254 T85 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 4255 T71 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 4255 T58 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 4255 T99 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 4261 T31 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 4262 T31 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 4262 T31 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 4326 T31 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 4328 T31 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:35158
[junit4:junit4] 2> 4329 T31 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 4329 T31 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 4330 T31 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty4-1364129516726
[junit4:junit4] 2> 4330 T31 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty4-1364129516726/solr.xml
[junit4:junit4] 2> 4331 T31 oasc.CoreContainer.<init> New CoreContainer 23052275
[junit4:junit4] 2> 4331 T31 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty4-1364129516726/'
[junit4:junit4] 2> 4332 T31 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty4-1364129516726/'
[junit4:junit4] 2> 4350 T31 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 4351 T31 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 4351 T31 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 4351 T31 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 4352 T31 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 4352 T31 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 4353 T31 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 4353 T31 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 4353 T31 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 4354 T31 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 4362 T31 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 4377 T31 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:38828/solr
[junit4:junit4] 2> 4377 T31 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 4378 T31 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:38828 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@5b268
[junit4:junit4] 2> 4379 T31 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 4379 T110 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:38828. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 4380 T110 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:38828, initiating session
[junit4:junit4] 2> 4380 T33 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:40105
[junit4:junit4] 2> 4381 T33 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:40105
[junit4:junit4] 2> 4382 T35 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9c747c6f000b with negotiated timeout 20000 for client /127.0.0.1:40105
[junit4:junit4] 2> 4382 T110 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:38828, sessionid = 0x13d9c747c6f000b, negotiated timeout = 20000
[junit4:junit4] 2> 4383 T111 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5b268 name:ZooKeeperConnection Watcher:127.0.0.1:38828 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 4384 T31 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 4386 T36 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d9c747c6f000b
[junit4:junit4] 2> 4387 T111 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 4387 T33 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:40105 which had sessionid 0x13d9c747c6f000b
[junit4:junit4] 2> 4387 T31 oaz.ZooKeeper.close Session: 0x13d9c747c6f000b closed
[junit4:junit4] 2> 4389 T31 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 4393 T31 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:38828/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@339c34
[junit4:junit4] 2> 4394 T112 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:38828. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 4394 T31 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 4395 T112 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:38828, initiating session
[junit4:junit4] 2> 4395 T33 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:40106
[junit4:junit4] 2> 4395 T33 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:40106
[junit4:junit4] 2> 4396 T35 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9c747c6f000c with negotiated timeout 20000 for client /127.0.0.1:40106
[junit4:junit4] 2> 4396 T112 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:38828, sessionid = 0x13d9c747c6f000c, negotiated timeout = 20000
[junit4:junit4] 2> 4397 T113 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@339c34 name:ZooKeeperConnection Watcher:127.0.0.1:38828/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 4397 T31 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 4398 T36 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c747c6f000c type:create cxid:0x1 zxid:0x48 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 4400 T36 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c747c6f000c type:create cxid:0x3 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 4403 T31 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 5408 T31 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35158_
[junit4:junit4] 2> 5410 T36 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c747c6f000c type:delete cxid:0xd zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:35158_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:35158_
[junit4:junit4] 2> 5412 T31 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:35158_
[junit4:junit4] 2> 5419 T53 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 5419 T99 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 5420 T71 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 5420 T58 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 5420 T113 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 5420 T85 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 5433 T31 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 5434 T31 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 5435 T31 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 5442 T31 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 5642 T65 oasc.CoreContainer.create Creating SolrCore 'unloadcollection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty1-1364129513183/unloadcollection1
[junit4:junit4] 2> 5644 T65 oasc.ZkController.createCollectionZkNode Check for collection zkNode:unloadcollection
[junit4:junit4] 2> 5646 T65 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:unloadcollection
[junit4:junit4] 2> 5648 T65 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 5650 T65 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 5652 T65 oascc.SolrZkClient.makePath makePath: /collections/unloadcollection
[junit4:junit4] 2> 5656 T65 oasc.ZkController.readConfigName Load collection config from:/collections/unloadcollection
[junit4:junit4] 2> 5659 T65 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty1-1364129513183/unloadcollection1/'
[junit4:junit4] 2> 5719 T65 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4] 2> 5765 T65 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 5767 T65 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 5808 T65 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 6259 T65 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 6270 T65 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 6273 T65 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 6305 T65 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 6311 T65 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 6316 T65 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 6319 T65 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 6319 T65 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 6320 T65 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 6322 T65 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 6322 T65 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 6323 T65 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 6323 T65 oasc.SolrCore.<init> [unloadcollection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty1-1364129513183/unloadcollection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1364129512471/1364129517911unloadcollection1_1n/
[junit4:junit4] 2> 6325 T65 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1e2550a
[junit4:junit4] 2> 6328 T65 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 6328 T65 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1364129512471/1364129517911unloadcollection1_1n
[junit4:junit4] 2> 6329 T65 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1364129512471/1364129517911unloadcollection1_1n/index/
[junit4:junit4] 2> 6329 T65 oasc.SolrCore.initIndex WARNING [unloadcollection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1364129512471/1364129517911unloadcollection1_1n/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 6330 T65 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1364129512471/1364129517911unloadcollection1_1n/index
[junit4:junit4] 2> 6332 T65 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@27b834 lockFactory=org.apache.lucene.store.NativeFSLockFactory@55ef6f),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 6332 T65 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 6345 T65 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 6346 T65 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 6347 T65 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 6348 T65 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 6349 T65 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 6349 T65 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 6349 T65 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 6350 T65 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 6351 T65 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 6353 T65 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 6356 T65 oass.SolrIndexSearcher.<init> Opening Searcher@244168 main
[junit4:junit4] 2> 6358 T65 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1364129512471/1364129517911unloadcollection1_1n/tlog
[junit4:junit4] 2> 6364 T65 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 6364 T65 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 6371 T115 oasc.SolrCore.registerSearcher [unloadcollection1] Registered new searcher Searcher@244168 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 6376 T65 oasc.ZkController.publish publishing core=unloadcollection1 state=down
[junit4:junit4] 2> 6703 T54 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 6705 T54 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "core":"unloadcollection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "node_name":"127.0.0.1:43934_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:43934",
[junit4:junit4] 2> "collection":"unloadcollection",
[junit4:junit4] 2> "shard":null}
[junit4:junit4] 2> 6706 T54 oasc.Overseer$ClusterStateUpdater.createCollection Create collection unloadcollection with numShards 1
[junit4:junit4] 2> 6721 T54 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 6739 T85 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 6739 T113 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 6739 T53 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 6739 T99 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 6739 T71 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 6739 T58 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 7379 T65 oasc.CoreContainer.registerCore registering core: unloadcollection1
[junit4:junit4] 2> 7380 T65 oasc.ZkController.register Register replica - core:unloadcollection1 address:http://127.0.0.1:43934 collection:unloadcollection shard:shard1
[junit4:junit4] 2> 7387 T65 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4] 2> 7394 T65 oascc.SolrZkClient.makePath makePath: /collections/unloadcollection/leader_elect/shard1/election
[junit4:junit4] 2> 7413 T36 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c747c6f0006 type:delete cxid:0x51 zxid:0x57 txntype:-1 reqpath:n/a Error Path:/solr/collections/unloadcollection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/unloadcollection/leaders
[junit4:junit4] 2> 7414 T65 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 7416 T36 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c747c6f0006 type:create cxid:0x52 zxid:0x58 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 7423 T65 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 7423 T65 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 7424 T65 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:43934/unloadcollection1/
[junit4:junit4] 2> 7425 T65 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 7426 T65 oasc.SyncStrategy.syncToMe http://127.0.0.1:43934/unloadcollection1/ has no replicas
[junit4:junit4] 2> 7427 T65 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:43934/unloadcollection1/
[junit4:junit4] 2> 7427 T65 oascc.SolrZkClient.makePath makePath: /collections/unloadcollection/leaders/shard1
[junit4:junit4] 2> 7439 T36 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c747c6f0006 type:create cxid:0x5d zxid:0x5c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 8250 T54 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 8269 T99 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 8269 T85 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 8269 T71 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 8269 T113 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 8269 T53 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 8270 T58 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 8298 T65 oasc.ZkController.register We are http://127.0.0.1:43934/unloadcollection1/ and leader is http://127.0.0.1:43934/unloadcollection1/
[junit4:junit4] 2> 8301 T65 oasc.ZkController.register No LogReplay needed for core=unloadcollection1 baseURL=http://127.0.0.1:43934
[junit4:junit4] 2> 8301 T65 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 8302 T65 oasc.ZkController.publish publishing core=unloadcollection1 state=active
[junit4:junit4] 2> 8306 T65 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 8309 T65 oasc.CoreContainer.persistFile Persisting cores config to /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty1-1364129513183/solr.xml
[junit4:junit4] 2> 8323 T65 oasc.SolrXMLSerializer.persistFile Persisting cores config to /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty1-1364129513183/solr.xml
[junit4:junit4] 2> 8326 T65 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {numShards=1&dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1364129512471/1364129517911unloadcollection1_1n&action=CREATE&name=unloadcollection1&version=2&wt=javabin&collection=unloadcollection} status=0 QTime=2687
[junit4:junit4] 2> 8357 T31 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 8359 T31 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:38828/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@d3d1d5
[junit4:junit4] 2> 8360 T31 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 8360 T116 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:38828. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 8361 T116 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:38828, initiating session
[junit4:junit4] 2> 8361 T33 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:40111
[junit4:junit4] 2> 8362 T33 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:40111
[junit4:junit4] 2> 8370 T35 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9c747c6f000d with negotiated timeout 10000 for client /127.0.0.1:40111
[junit4:junit4] 2> 8370 T116 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:38828, sessionid = 0x13d9c747c6f000d, negotiated timeout = 10000
[junit4:junit4] 2> 8371 T117 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d3d1d5 name:ZooKeeperConnection Watcher:127.0.0.1:38828/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 8372 T31 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 8375 T31 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 8379 T31 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 8381 T31 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 8392 T79 oasc.CoreContainer.create Creating SolrCore 'unloadcollection2' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty2-1364129514377/unloadcollection2
[junit4:junit4] 2> 8393 T79 oasc.ZkController.createCollectionZkNode Check for collection zkNode:unloadcollection
[junit4:junit4] 2> 8394 T79 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 8395 T79 oasc.ZkController.readConfigName Load collection config from:/collections/unloadcollection
[junit4:junit4] 2> 8398 T79 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty2-1364129514377/unloadcollection2/'
[junit4:junit4] 2> 8451 T79 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4] 2> 8489 T79 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 8491 T79 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 8518 T79 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 8913 T79 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 8926 T79 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 8929 T79 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 8971 T79 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 8980 T79 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 8988 T79 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 8991 T79 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 8993 T79 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 8993 T79 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 8996 T79 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 8997 T79 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 8998 T79 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 8999 T79 oasc.SolrCore.<init> [unloadcollection2] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty2-1364129514377/unloadcollection2/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1364129512471/1364129520849unloadcollection1_2n/
[junit4:junit4] 2> 8999 T79 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1e2550a
[junit4:junit4] 2> 9001 T79 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 9003 T79 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1364129512471/1364129520849unloadcollection1_2n
[junit4:junit4] 2> 9003 T79 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1364129512471/1364129520849unloadcollection1_2n/index/
[junit4:junit4] 2> 9004 T79 oasc.SolrCore.initIndex WARNING [unloadcollection2] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1364129512471/1364129520849unloadcollection1_2n/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 9005 T79 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1364129512471/1364129520849unloadcollection1_2n/index
[junit4:junit4] 2> 9009 T79 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4c3d04 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1555268),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 9009 T79 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 9014 T79 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 9015 T79 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 9017 T79 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 9018 T79 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 9020 T79 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 9020 T79 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 9021 T79 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 9023 T79 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 9024 T79 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 9027 T79 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 9030 T79 oass.SolrIndexSearcher.<init> Opening Searcher@8d47a0 main
[junit4:junit4] 2> 9030 T79 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1364129512471/1364129520849unloadcollection1_2n/tlog
[junit4:junit4] 2> 9031 T79 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 9031 T79 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 9040 T118 oasc.SolrCore.registerSearcher [unloadcollection2] Registered new searcher Searcher@8d47a0 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 9047 T79 oasc.ZkController.publish publishing core=unloadcollection2 state=down
[junit4:junit4] 2> 9048 T79 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 9788 T54 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 9790 T54 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:43934__unloadcollection1",
[junit4:junit4] 2> "core":"unloadcollection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "node_name":"127.0.0.1:43934_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:43934",
[junit4:junit4] 2> "collection":"unloadcollection",
[junit4:junit4] 2> "shard":"shard1"}
[junit4:junit4] 2> 9800 T54 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "core":"unloadcollection2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "node_name":"127.0.0.1:45516_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:45516",
[junit4:junit4] 2> "collection":"unloadcollection",
[junit4:junit4] 2> "shard":null}
[junit4:junit4] 2> 9801 T54 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 9801 T54 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 9811 T53 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 9811 T99 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 9812 T71 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 9811 T113 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 9811 T58 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 9811 T117 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 9813 T85 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 10050 T79 oasc.CoreContainer.registerCore registering core: unloadcollection2
[junit4:junit4] 2> 10051 T79 oasc.ZkController.register Register replica - core:unloadcollection2 address:http://127.0.0.1:45516 collection:unloadcollection shard:shard1
[junit4:junit4] 2> 10061 T79 oasc.ZkController.register We are http://127.0.0.1:45516/unloadcollection2/ and leader is http://127.0.0.1:43934/unloadcollection1/
[junit4:junit4] 2> 10062 T79 oasc.ZkController.register No LogReplay needed for core=unloadcollection2 baseURL=http://127.0.0.1:45516
[junit4:junit4] 2> 10062 T79 oasc.ZkController.checkRecovery Core needs to recover:unloadcollection2
[junit4:junit4] 2> 10063 T79 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C1 name=unloadcollection2 org.apache.solr.core.SolrCore@b0cace url=http://127.0.0.1:45516/unloadcollection2 node=127.0.0.1:45516_ C1_STATE=coll:unloadcollection core:unloadcollection2 props:{state=down, core=unloadcollection2, node_name=127.0.0.1:45516_, base_url=http://127.0.0.1:45516, collection=unloadcollection}
[junit4:junit4] 2> 10068 T119 C1 P45516 oasc.RecoveryStrategy.run Starting recovery process. core=unloadcollection2 recoveringAfterStartup=true
[junit4:junit4] 2> 10069 T119 C1 P45516 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 10069 T79 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 10070 T119 C1 P45516 oasc.ZkController.publish publishing core=unloadcollection2 state=recovering
[junit4:junit4] 2> 10071 T119 C1 P45516 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 10072 T79 oasc.CoreContainer.persistFile Persisting cores config to /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty2-1364129514377/solr.xml
[junit4:junit4] 2> 10073 T119 C1 P45516 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 10079 T79 oasc.SolrXMLSerializer.persistFile Persisting cores config to /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty2-1364129514377/solr.xml
[junit4:junit4] 2> 10080 T79 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1364129512471/1364129520849unloadcollection1_2n&action=CREATE&name=unloadcollection2&version=2&wt=javabin&collection=unloadcollection} status=0 QTime=1688
[junit4:junit4] 2> 10082 T31 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 10084 T31 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: unloadcollection failOnTimeout:true timeout (sec):230
[junit4:junit4] 2> 10085 T31 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 11089 T31 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 11330 T54 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 11332 T54 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:45516__unloadcollection2",
[junit4:junit4] 2> "core":"unloadcollection2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "node_name":"127.0.0.1:45516_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:45516",
[junit4:junit4] 2> "collection":"unloadcollection",
[junit4:junit4] 2> "shard":"shard1"}
[junit4:junit4] 2> 11440 T71 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 11440 T113 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 11441 T99 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 11441 T53 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 11440 T117 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 11440 T85 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 11440 T58 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 12081 T65 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {state=recovering&core=unloadcollection1&onlyIfLeader=true&action=PREPRECOVERY&version=2&coreNodeName=127.0.0.1:45516__unloadcollection2&checkLive=true&wt=javabin&nodeName=127.0.0.1:45516_} status=0 QTime=2002
[junit4:junit4] 2> 12093 T31 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 13096 T31 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C1_STATE=coll:unloadcollection core:unloadcollection2 props:{state=recovering, core=unloadcollection2, node_name=127.0.0.1:45516_, base_url=http://127.0.0.1:45516, collection=unloadcollection, shard=shard1}
[junit4:junit4] 2> 14083 T119 C1 P45516 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:43934/unloadcollection1/ core=unloadcollection2 - recoveringAfterStartup=true
[junit4:junit4] 2> 14088 T119 C1 P45516 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4] 2> 14100 T31 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 14101 T119 C1 P45516 oasu.PeerSync.sync PeerSync: core=unloadcollection2 url=http://127.0.0.1:45516 START replicas=[http://127.0.0.1:43934/unloadcollection1/] nUpdates=100
[junit4:junit4] 2> 14106 T119 C1 P45516 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 14106 T119 C1 P45516 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=unloadcollection2
[junit4:junit4] 2> 14107 T119 C1 P45516 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=unloadcollection2
[junit4:junit4] 2> 14107 T119 C1 P45516 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=unloadcollection2
[junit4:junit4] 2> 14107 T119 C1 P45516 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 14107 T119 C1 P45516 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:43934/unloadcollection1/. core=unloadcollection2
[junit4:junit4] 2> 14107 T119 C1 P45516 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C2 name=unloadcollection1 org.apache.solr.core.SolrCore@946ce8 url=http://127.0.0.1:43934/unloadcollection1 node=127.0.0.1:43934_ C2_STATE=coll:unloadcollection core:unloadcollection1 props:{state=active, core=unloadcollection1, node_name=127.0.0.1:43934_, base_url=http://127.0.0.1:43934, collection=unloadcollection, shard=shard1, leader=true}
[junit4:junit4] 2> 14125 T66 C2 P43934 REQ /get {version=2&wt=javabin&getVersions=100&distrib=false&qt=/get} status=0 QTime=3
[junit4:junit4] 2> 14133 T67 C2 P43934 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 14136 T67 C2 P43934 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@27b834 lockFactory=org.apache.lucene.store.NativeFSLockFactory@55ef6f),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 14137 T67 C2 P43934 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 14138 T67 C2 P43934 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@27b834 lockFactory=org.apache.lucene.store.NativeFSLockFactory@55ef6f),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@27b834 lockFactory=org.apache.lucene.store.NativeFSLockFactory@55ef6f),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> 14138 T67 C2 P43934 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4] 2> 14139 T67 C2 P43934 oass.SolrIndexSearcher.<init> Opening Searcher@9877df realtime
[junit4:junit4] 2> 14139 T67 C2 P43934 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 14139 T67 C2 P43934 /update {commit=true&softCommit=false&version=2&waitSearcher=true&commit_end_point=true&wt=javabin&openSearcher=false} {commit=} 0 18
[junit4:junit4] 2> 14170 T119 C1 P45516 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 14170 T119 C1 P45516 oascsi.HttpClientUtil.createClient Creating new http client, config:connTimeout=5000&socketTimeout=20000&allowCompression=false&maxConnections=10000&maxConnectionsPerHost=10000
[junit4:junit4] 2> 14175 T119 C1 P45516 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 14210 T65 C2 P43934 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 14211 T65 C2 P43934 REQ /replication {version=2&command=indexversion&wt=javabin&qt=/replication} status=0 QTime=33
[junit4:junit4] 2> 14212 T119 C1 P45516 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 14212 T119 C1 P45516 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 14213 T119 C1 P45516 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 14216 T66 C2 P43934 REQ /replication {generation=2&version=2&command=filelist&wt=javabin&qt=/replication} status=0 QTime=1
[junit4:junit4] 2> 14217 T119 C1 P45516 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 14218 T119 C1 P45516 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1364129512471/1364129520849unloadcollection1_2n/index.20130324085206678
[junit4:junit4] 2> 14218 T119 C1 P45516 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@da78d5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@f65578) fullCopy=false
[junit4:junit4] 2> 14224 T67 C2 P43934 REQ /replication {generation=2&checksum=true&file=segments_2&command=filecontent&wt=filestream&qt=/replication} status=0 QTime=1
[junit4:junit4] 2> 14228 T119 C1 P45516 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 14230 T119 C1 P45516 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 14231 T119 C1 P45516 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=unloadcollection2
[junit4:junit4] 2> 14233 T119 C1 P45516 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4c3d04 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1555268),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4c3d04 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1555268),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> 14234 T119 C1 P45516 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4] 2> 14234 T119 C1 P45516 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 14235 T119 C1 P45516 oass.SolrIndexSearcher.<init> Opening Searcher@1c79e main
[junit4:junit4] 2> 14239 T118 oasc.SolrCore.registerSearcher [unloadcollection2] Registered new searcher Searcher@1c79e main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 14240 T119 C1 P45516 oasc.RecoveryStrategy.replay No replay needed. core=unloadcollection2
[junit4:junit4] 2> 14240 T119 C1 P45516 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=unloadcollection2
[junit4:junit4] 2> 14240 T119 C1 P45516 oasc.ZkController.publish publishing core=unloadcollection2 state=active
[junit4:junit4] 2> 14241 T119 C1 P45516 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 14243 T119 C1 P45516 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=unloadcollection2
[junit4:junit4] 2> 14456 T54 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 14459 T54 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:45516__unloadcollection2",
[junit4:junit4] 2> "core":"unloadcollection2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "node_name":"127.0.0.1:45516_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:45516",
[junit4:junit4] 2> "collection":"unloadcollection",
[junit4:junit4] 2> "shard":"shard1"}
[junit4:junit4] 2> 14470 T117 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 14471 T71 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 14471 T85 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 14470 T58 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 14471 T53 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 14471 T99 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 14470 T113 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 15103 T31 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 15105 T31 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: unloadcollection
[junit4:junit4] 2> 15105 T31 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 15122 T94 oasc.CoreContainer.create Creating SolrCore 'unloadcollection3' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty3-1364129515557/unloadcollection3
[junit4:junit4] 2> 15122 T94 oasc.ZkController.createCollectionZkNode Check for collection zkNode:unloadcollection
[junit4:junit4] 2> 15124 T94 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 15125 T94 oasc.ZkController.readConfigName Load collection config from:/collections/unloadcollection
[junit4:junit4] 2> 15128 T94 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty3-1364129515557/unloadcollection3/'
[junit4:junit4] 2> 15170 T94 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4] 2> 15208 T94 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 15209 T94 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 15231 T94 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 15665 T94 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 15675 T94 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 15678 T94 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 15708 T94 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 15713 T94 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 15717 T94 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 15719 T94 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 15719 T94 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 15720 T94 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 15722 T94 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 15722 T94 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 15722 T94 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 15723 T94 oasc.SolrCore.<init> [unloadcollection3] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty3-1364129515557/unloadcollection3/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1364129512471/1364129527575unloadcollection_3n/
[junit4:junit4] 2> 15723 T94 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1e2550a
[junit4:junit4] 2> 15724 T94 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 15725 T94 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1364129512471/1364129527575unloadcollection_3n
[junit4:junit4] 2> 15725 T94 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1364129512471/1364129527575unloadcollection_3n/index/
[junit4:junit4] 2> 15725 T94 oasc.SolrCore.initIndex WARNING [unloadcollection3] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1364129512471/1364129527575unloadcollection_3n/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 15726 T94 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1364129512471/1364129527575unloadcollection_3n/index
[junit4:junit4] 2> 15727 T94 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@fefd95 lockFactory=org.apache.lucene.store.NativeFSLockFactory@54575e),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 15728 T94 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 15730 T94 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 15731 T94 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 15731 T94 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 15732 T94 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 15733 T94 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 15733 T94 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 15733 T94 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 15734 T94 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 15735 T94 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 15740 T94 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 15746 T94 oass.SolrIndexSearcher.<init> Opening Searcher@1680587 main
[junit4:junit4] 2> 15746 T94 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1364129512471/1364129527575unloadcollection_3n/tlog
[junit4:junit4] 2> 15747 T94 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 15747 T94 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 15752 T122 oasc.SolrCore.registerSearcher [unloadcollection3] Registered new searcher Searcher@1680587 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 15755 T94 oasc.ZkController.publish publishing core=unloadcollection3 state=down
[junit4:junit4] 2> 15755 T94 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 15980 T54 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 15981 T54 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "core":"unloadcollection3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "node_name":"127.0.0.1:44754_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:44754",
[junit4:junit4] 2> "collection":"unloadcollection",
[junit4:junit4] 2> "shard":null}
[junit4:junit4] 2> 15981 T54 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 15982 T54 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 16081 T53 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 16082 T117 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 16082 T85 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 16082 T113 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 16082 T99 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 16082 T71 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 16082 T58 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 16758 T94 oasc.CoreContainer.registerCore registering core: unloadcollection3
[junit4:junit4] 2> 16759 T94 oasc.ZkController.register Register replica - core:unloadcollection3 address:http://127.0.0.1:44754 collection:unloadcollection shard:shard1
[junit4:junit4] 2> 16765 T94 oasc.ZkController.register We are http://127.0.0.1:44754/unloadcollection3/ and leader is http://127.0.0.1:43934/unloadcollection1/
[junit4:junit4] 2> 16766 T94 oasc.ZkController.register No LogReplay needed for core=unloadcollection3 baseURL=http://127.0.0.1:44754
[junit4:junit4] 2> 16766 T94 oasc.ZkController.checkRecovery Core needs to recover:unloadcollection3
[junit4:junit4] 2> 16766 T94 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C3 name=unloadcollection3 org.apache.solr.core.SolrCore@88dbd9 url=http://127.0.0.1:44754/unloadcollection3 node=127.0.0.1:44754_ C3_STATE=coll:unloadcollection core:unloadcollection3 props:{state=down, core=unloadcollection3, node_name=127.0.0.1:44754_, base_url=http://127.0.0.1:44754, collection=unloadcollection}
[junit4:junit4] 2> 16767 T123 C3 P44754 oasc.RecoveryStrategy.run Starting recovery process. core=unloadcollection3 recoveringAfterStartup=true
[junit4:junit4] 2> 16768 T123 C3 P44754 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 16769 T94 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 16769 T123 C3 P44754 oasc.ZkController.publish publishing core=unloadcollection3 state=recovering
[junit4:junit4] 2> 16770 T123 C3 P44754 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 16771 T94 oasc.CoreContainer.persistFile Persisting cores config to /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty3-1364129515557/solr.xml
[junit4:junit4] 2> 16785 T94 oasc.SolrXMLSerializer.persistFile Persisting cores config to /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.UnloadDistributedZkTest-jetty3-1364129515557/solr.xml
[junit4:junit4] 2> 16786 T94 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1364129512471/1364129527575unloadcollection_3n&action=CREATE&name=unloadcollection3&version=2&wt=javabin&collection=unloadcollection} status=0 QTime=1665
[junit4:junit4] 2> 16787 T31 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: unloadcollection failOnTimeout:true timeout (sec):230
[junit4:junit4] 2> 16791 T123 C3 P44754 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 16791 T31 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 17594 T54 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 17595 T54 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:44754__unloadcollection3",
[junit4:junit4] 2> "core":"unloadcollection3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "node_name":"127.0.0.1:44754_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:44754",
[junit4:junit4] 2> "collection":"unloadcollection",
[junit4:junit4] 2> "shard":"shard1"}
[junit4:junit4] 2> 17602 T85 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 17602 T71 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 17603 T58 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 17603 T117 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 17603 T113 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 17603 T99 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 17603 T53 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 17798 T31 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 17803 T64 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {state=recovering&core=unloadcollection1&onlyIfLeader=true&action=PREPRECOVERY&version=2&coreNodeName=127.0.0.1:44754__unloadcollection3&checkLive=true&wt=javabin&nodeName=127.0.0.1:44754_} status=0 QTime=1000
[junit4:junit4] 2> 18804 T31 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C3_STATE=coll:unloadcollection core:unloadcollection3 props:{state=recovering, core=unloadcollection3, node_name=127.0.0.1:44754_, base_url=http://127.0.0.1:44754, collection=unloadcollection, shard=shard1}
[junit4:junit4] 2> 19804 T123 C3 P44754 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:43934/unloadcollection1/ core=unloadcollection3 - recoveringAfterStartup=true
[junit4:junit4] 2> 19805 T123 C3 P44754 oasu.PeerSync.sync PeerSync: core=unloadcollection3 url=http://127.0.0.1:44754 START replicas=[http://127.0.0.1:43934/unloadcollection1/] nUpdates=100
[junit4:junit4] 2> 19805 T123 C3 P44754 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 19806 T123 C3 P44754 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=unloadcollection3
[junit4:junit4] 2> 19807 T123 C3 P44754 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=unloadcollection3
[junit4:junit4] 2> 19807 T123 C3 P44754 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=unloadcollection3
[junit4:junit4] 2> 19807 T123 C3 P44754 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 19807 T123 C3 P44754 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:43934/unloadcollection1/. core=unloadcollection3
[junit4:junit4] 2> 19807 T31 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 19807 T123 C3 P44754 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 19809 T64 C2 P43934 REQ /get {version=2&wt=javabin&getVersions=100&distrib=false&qt=/get} status=0 QTime=0
[junit4:junit4] 2> 19815 T65 C2 P43934 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 19817 T65 C2 P43934 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@27b834 lockFactory=org.apache.lucene.store.NativeFSLockFactory@55ef6f),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@27b834 lockFactory=org.apache.lucene.store.NativeFSLockFactory@55ef6f),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4] 2> 19817 T65 C2 P43934 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4] 2> 19818 T65 C2 P43934 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 19819 T65 C2 P43934 /update {commit=true&softCommit=false&version=2&waitSearcher=true&commit_end_point=true&wt=javabin&openSearcher=false} {commit=} 0 5
[junit4:junit4] 2> 19820 T123 C3 P44754 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 19821 T123 C3 P44754 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 19826 T67 C2 P43934 REQ /replication {version=2&command=indexversion&wt=javabin&qt=/replication} status=0 QTime=1
[junit4:junit4] 2> 19827 T123 C3 P44754 oash.SnapPuller.fetchLatestIndex Master's generation: 3
[junit4:junit4] 2> 19827 T123 C3 P44754 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 19827 T123 C3 P44754 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 19830 T64 C2 P43934 REQ /replication {generation=3&version=2&command=filelist&wt=javabin&qt=/replication} status=0 QTime=0
[junit4:junit4] 2> 19831 T123 C3 P44754 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 19832 T123 C3 P44754 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-UnloadDistributedZkTest-1364129512471/1364129527575unloadcollection_3n/index.20130324085212292
[junit4:junit4] 2> 19832 T123 C3 P44754 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@1801ff4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@12da36a) fullCopy=false
[junit4:junit4] 2> 19836 T65 C2 P43934 REQ /replication {generation=3&checksum=true&file=segments_3&command=filecontent&wt=filestream&qt=/replication} status=0 QTime=0
[junit4:junit4] 2> 19838 T123 C3 P44754 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 19840 T123 C3 P44754 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 19841 T123 C3 P44754 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=unloadcollection3
[junit4:junit4] 2> 19844 T123 C3 P44754 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@fefd95 lockFactory=org.apache.lucene.store.NativeFSLockFactory@54575e),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@fefd95 lockFactory=org.apache.lucene.store.NativeFSLockFactory@54575e),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4] 2> 19845 T123 C3 P44754 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4] 2> 19846 T123 C3 P44754 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 19847 T123 C3 P44754 oass.SolrIndexSearcher.<init> Opening Searcher@18eb64d main
[junit4:junit4] 2> 19849 T122 oasc.SolrCore.registerSearcher [unloadcollection3] Registered new searcher Searcher@18eb64d main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4] 2> 19851 T123 C3 P44754 oasc.RecoveryStrategy.replay No replay needed. core=unloadcollection3
[junit4:junit4] 2> 19852 T123 C3 P44754 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=unloadcollection3
[junit4:junit4] 2> 19852 T123 C3 P44754 oasc.ZkController.publish publishing core=unloadcollection3 state=active
[junit4:junit4] 2> 19853 T123 C3 P44754 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 19856 T123 C3 P44754 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=unloadcollection3
[junit4:junit4] 2> 20615 T54 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 20616 T54 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:44754__unloadcollection3",
[junit4:junit4] 2> "core":"unloadcollection3",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "node_name":"127.0.0.1:44754_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:44754",
[junit4:junit4] 2> "collection":"unloadcollection",
[junit4:junit4] 2> "shard":"shard1"}
[junit4:junit4] 2> 20740 T113 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 20740 T53 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 20740 T58 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 20740 T117 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 20740 T85 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 20740 T99 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 20740 T71 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 20810 T31 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 20811 T31 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: unloadcollection
[junit4:junit4] 2> 20812 T31 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> C3_STATE=coll:unloadcollection core:unloadcollection3 props:{state=active, core=unloadcollection3, node_name=127.0.0.1:44754_, base_url=http://127.0.0.1:44754, collection=unloadcollection, shard=shard1}
[junit4:junit4] 2> 20998 T94 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[20 (1430393489600806912)]} 0 5
[junit4:junit4] 2> C1_STATE=coll:unloadcollection core:unloadcollection2 props:{state=active, core=unloadcollection2, node_name=127.0.0.1:45516_, base_url=http://127.0.0.1:45516, collection=unloadcollection, shard=shard1}
[junit4:junit4] 2> 21002 T80 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[20 (1430393489600806912)]} 0 4
[junit4:junit4] 2> 21003 T67 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[20 (1430393489600806912)]} 0 78
[junit4:junit4] 2> 21004 T92 C3 P44754 /update {version=2&wt=javabin} {add=[20]} 0 170
[junit4:junit4] 2> 21020 T95 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[21 (1430393489688887296)]} 0 4
[junit4:junit4] 2> 21020 T79 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[21 (1430393489688887296)]} 0 4
[junit4:junit4] 2> 21021 T64 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[21 (1430393489688887296)]} 0 11
[junit4:junit4] 2> 21022 T93 C3 P44754 /update {version=2&wt=javabin} {add=[21]} 0 15
[junit4:junit4] 2> 21035 T81 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[22 (1430393489706713088)]} 0 3
[junit4:junit4] 2> 21035 T92 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[22 (1430393489706713088)]} 0 3
[junit4:junit4] 2> 21036 T65 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[22 (1430393489706713088)]} 0 9
[junit4:junit4] 2> 21037 T94 C3 P44754 /update {version=2&wt=javabin} {add=[22]} 0 11
[junit4:junit4] 2> 21049 T93 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[23 (1430393489722441728)]} 0 3
[junit4:junit4] 2> 21049 T78 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[23 (1430393489722441728)]} 0 3
[junit4:junit4] 2> 21050 T67 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[23 (1430393489722441728)]} 0 8
[junit4:junit4] 2> 21051 T95 C3 P44754 /update {version=2&wt=javabin} {add=[23]} 0 12
[junit4:junit4] 2> 21062 T80 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[24 (1430393489737121792)]} 0 2
[junit4:junit4] 2> 21062 T94 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[24 (1430393489737121792)]} 0 2
[junit4:junit4] 2> 21063 T66 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[24 (1430393489737121792)]} 0 7
[junit4:junit4] 2> 21064 T92 C3 P44754 /update {version=2&wt=javabin} {add=[24]} 0 11
[junit4:junit4] 2> 21077 T79 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[25 (1430393489750753280)]} 0 3
[junit4:junit4] 2> 21077 T95 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[25 (1430393489750753280)]} 0 3
[junit4:junit4] 2> 21078 T64 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[25 (1430393489750753280)]} 0 9
[junit4:junit4] 2> 21079 T93 C3 P44754 /update {version=2&wt=javabin} {add=[25]} 0 13
[junit4:junit4] 2> 21090 T92 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[26 (1430393489766481920)]} 0 2
[junit4:junit4] 2> 21090 T81 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[26 (1430393489766481920)]} 0 2
[junit4:junit4] 2> 21091 T65 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[26 (1430393489766481920)]} 0 7
[junit4:junit4] 2> 21092 T94 C3 P44754 /update {version=2&wt=javabin} {add=[26]} 0 11
[junit4:junit4] 2> 21103 T93 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[27 (1430393489780113408)]} 0 2
[junit4:junit4] 2> 21103 T78 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[27 (1430393489780113408)]} 0 2
[junit4:junit4] 2> 21104 T67 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[27 (1430393489780113408)]} 0 7
[junit4:junit4] 2> 21105 T95 C3 P44754 /update {version=2&wt=javabin} {add=[27]} 0 11
[junit4:junit4] 2> 21118 T80 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[28 (1430393489794793472)]} 0 2
[junit4:junit4] 2> 21119 T94 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[28 (1430393489794793472)]} 0 3
[junit4:junit4] 2> 21120 T66 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[28 (1430393489794793472)]} 0 9
[junit4:junit4] 2> 21121 T92 C3 P44754 /update {version=2&wt=javabin} {add=[28]} 0 13
[junit4:junit4] 2> 21134 T79 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[29 (1430393489811570688)]} 0 3
[junit4:junit4] 2> 21134 T95 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[29 (1430393489811570688)]} 0 3
[junit4:junit4] 2> 21135 T64 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[29 (1430393489811570688)]} 0 8
[junit4:junit4] 2> 21136 T93 C3 P44754 /update {version=2&wt=javabin} {add=[29]} 0 12
[junit4:junit4] 2> 21150 T92 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[30 (1430393489827299328)]} 0 3
[junit4:junit4] 2> 21150 T81 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[30 (1430393489827299328)]} 0 3
[junit4:junit4] 2> 21151 T65 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[30 (1430393489827299328)]} 0 10
[junit4:junit4] 2> 21152 T94 C3 P44754 /update {version=2&wt=javabin} {add=[30]} 0 14
[junit4:junit4] 2> 21164 T93 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[31 (1430393489844076544)]} 0 2
[junit4:junit4] 2> 21165 T78 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[31 (1430393489844076544)]} 0 2
[junit4:junit4] 2> 21166 T67 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[31 (1430393489844076544)]} 0 9
[junit4:junit4] 2> 21167 T95 C3 P44754 /update {version=2&wt=javabin} {add=[31]} 0 13
[junit4:junit4] 2> 21179 T94 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[32 (1430393489859805184)]} 0 2
[junit4:junit4] 2> 21179 T80 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[32 (1430393489859805184)]} 0 2
[junit4:junit4] 2> 21180 T66 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[32 (1430393489859805184)]} 0 7
[junit4:junit4] 2> 21181 T92 C3 P44754 /update {version=2&wt=javabin} {add=[32]} 0 11
[junit4:junit4] 2> 21195 T95 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[33 (1430393489874485248)]} 0 3
[junit4:junit4] 2> 21195 T79 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[33 (1430393489874485248)]} 0 2
[junit4:junit4] 2> 21196 T64 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[33 (1430393489874485248)]} 0 9
[junit4:junit4] 2> 21197 T93 C3 P44754 /update {version=2&wt=javabin} {add=[33]} 0 13
[junit4:junit4] 2> 21211 T81 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[34 (1430393489891262464)]} 0 2
[junit4:junit4] 2> 21213 T95 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[34 (1430393489891262464)]} 0 2
[junit4:junit4] 2> 21214 T65 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[34 (1430393489891262464)]} 0 11
[junit4:junit4] 2> 21215 T92 C3 P44754 /update {version=2&wt=javabin} {add=[34]} 0 15
[junit4:junit4] 2> 21229 T94 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[35 (1430393489911185408)]} 0 3
[junit4:junit4] 2> 21229 T80 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[35 (1430393489911185408)]} 0 3
[junit4:junit4] 2> 21231 T67 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[35 (1430393489911185408)]} 0 10
[junit4:junit4] 2> 21232 T93 C3 P44754 /update {version=2&wt=javabin} {add=[35]} 0 14
[junit4:junit4] 2> 21246 T92 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[36 (1430393489929011200)]} 0 3
[junit4:junit4] 2> 21246 T79 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[36 (1430393489929011200)]} 0 3
[junit4:junit4] 2> 21247 T66 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[36 (1430393489929011200)]} 0 8
[junit4:junit4] 2> 21248 T95 C3 P44754 /update {version=2&wt=javabin} {add=[36]} 0 12
[junit4:junit4] 2> 21260 T93 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[37 (1430393489945788416)]} 0 2
[junit4:junit4] 2> 21261 T78 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[37 (1430393489945788416)]} 0 2
[junit4:junit4] 2> 21262 T64 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[37 (1430393489945788416)]} 0 8
[junit4:junit4] 2> 21263 T94 C3 P44754 /update {version=2&wt=javabin} {add=[37]} 0 12
[junit4:junit4] 2> 21275 T95 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[38 (1430393489960468480)]} 0 3
[junit4:junit4] 2> 21275 T81 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[38 (1430393489960468480)]} 0 3
[junit4:junit4] 2> 21276 T65 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[38 (1430393489960468480)]} 0 8
[junit4:junit4] 2> 21277 T92 C3 P44754 /update {version=2&wt=javabin} {add=[38]} 0 11
[junit4:junit4] 2> 21292 T94 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[39 (1430393489975148544)]} 0 2
[junit4:junit4] 2> 21294 T80 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[39 (1430393489975148544)]} 0 3
[junit4:junit4] 2> 21295 T67 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[39 (1430393489975148544)]} 0 12
[junit4:junit4] 2> 21296 T93 C3 P44754 /update {version=2&wt=javabin} {add=[39]} 0 16
[junit4:junit4] 2> 21308 T92 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[40 (1430393489994022912)]} 0 3
[junit4:junit4] 2> 21308 T78 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[40 (1430393489994022912)]} 0 1
[junit4:junit4] 2> 21309 T66 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[40 (1430393489994022912)]} 0 8
[junit4:junit4] 2> 21310 T95 C3 P44754 /update {version=2&wt=javabin} {add=[40]} 0 12
[junit4:junit4] 2> 21321 T93 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[41 (1430393490008702976)]} 0 2
[junit4:junit4] 2> 21322 T81 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[41 (1430393490008702976)]} 0 3
[junit4:junit4] 2> 21323 T64 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[41 (1430393490008702976)]} 0 8
[junit4:junit4] 2> 21323 T94 C3 P44754 /update {version=2&wt=javabin} {add=[41]} 0 11
[junit4:junit4] 2> 21334 T95 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[42 (1430393490023383040)]} 0 1
[junit4:junit4] 2> 21335 T80 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[42 (1430393490023383040)]} 0 2
[junit4:junit4] 2> 21336 T65 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[42 (1430393490023383040)]} 0 7
[junit4:junit4] 2> 21337 T92 C3 P44754 /update {version=2&wt=javabin} {add=[42]} 0 11
[junit4:junit4] 2> 21348 T94 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[43 (1430393490037014528)]} 0 2
[junit4:junit4] 2> 21349 T79 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[43 (1430393490037014528)]} 0 2
[junit4:junit4] 2> 21349 T67 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[43 (1430393490037014528)]} 0 7
[junit4:junit4] 2> 21350 T93 C3 P44754 /update {version=2&wt=javabin} {add=[43]} 0 11
[junit4:junit4] 2> 21361 T78 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[44 (1430393490051694592)]} 0 2
[junit4:junit4] 2> 21362 T92 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[44 (1430393490051694592)]} 0 3
[junit4:junit4] 2> 21363 T66 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[44 (1430393490051694592)]} 0 8
[junit4:junit4] 2> 21364 T95 C3 P44754 /update {version=2&wt=javabin} {add=[44]} 0 11
[junit4:junit4] 2> 21375 T93 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[45 (1430393490065326080)]} 0 1
[junit4:junit4] 2> 21375 T81 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[45 (1430393490065326080)]} 0 1
[junit4:junit4] 2> 21377 T64 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[45 (1430393490065326080)]} 0 8
[junit4:junit4] 2> 21377 T94 C3 P44754 /update {version=2&wt=javabin} {add=[45]} 0 11
[junit4:junit4] 2> 21389 T95 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[46 (1430393490080006144)]} 0 2
[junit4:junit4] 2> 21389 T80 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[46 (1430393490080006144)]} 0 2
[junit4:junit4] 2> 21391 T65 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[46 (1430393490080006144)]} 0 8
[junit4:junit4] 2> 21392 T92 C3 P44754 /update {version=2&wt=javabin} {add=[46]} 0 12
[junit4:junit4] 2> 21403 T94 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[47 (1430393490094686208)]} 0 2
[junit4:junit4] 2> 21403 T79 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[47 (1430393490094686208)]} 0 1
[junit4:junit4] 2> 21404 T67 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[47 (1430393490094686208)]} 0 7
[junit4:junit4] 2> 21405 T93 C3 P44754 /update {version=2&wt=javabin} {add=[47]} 0 11
[junit4:junit4] 2> 21418 T95 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[48 (1430393490110414848)]} 0 2
[junit4:junit4] 2> 21418 T78 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[48 (1430393490110414848)]} 0 2
[junit4:junit4] 2> 21419 T66 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[48 (1430393490110414848)]} 0 7
[junit4:junit4] 2> 21420 T92 C3 P44754 /update {version=2&wt=javabin} {add=[48]} 0 11
[junit4:junit4] 2> 21431 T93 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[49 (1430393490124046336)]} 0 1
[junit4:junit4] 2> 21432 T81 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[49 (1430393490124046336)]} 0 2
[junit4:junit4] 2> 21433 T65 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[49 (1430393490124046336)]} 0 8
[junit4:junit4] 2> 21434 T94 C3 P44754 /update {version=2&wt=javabin} {add=[49]} 0 11
[junit4:junit4] 2> 21447 T92 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[50 (1430393490139774976)]} 0 2
[junit4:junit4] 2> 21447 T80 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[50 (1430393490139774976)]} 0 2
[junit4:junit4] 2> 21448 T67 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[50 (1430393490139774976)]} 0 8
[junit4:junit4] 2> 21449 T95 C3 P44754 /update {version=2&wt=javabin} {add=[50]} 0 12
[junit4:junit4] 2> 21461 T94 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[51 (1430393490154455040)]} 0 2
[junit4:junit4] 2> 21461 T79 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[51 (1430393490154455040)]} 0 2
[junit4:junit4] 2> 21462 T66 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[51 (1430393490154455040)]} 0 8
[junit4:junit4] 2> 21462 T93 C3 P44754 /update {version=2&wt=javabin} {add=[51]} 0 11
[junit4:junit4] 2> 21473 T95 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[52 (1430393490169135104)]} 0 2
[junit4:junit4] 2> 21473 T78 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[52 (1430393490169135104)]} 0 2
[junit4:junit4] 2> 21475 T65 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[52 (1430393490169135104)]} 0 7
[junit4:junit4] 2> 21476 T92 C3 P44754 /update {version=2&wt=javabin} {add=[52]} 0 11
[junit4:junit4] 2> 21488 T81 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[53 (1430393490183815168)]} 0 3
[junit4:junit4] 2> 21488 T93 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[53 (1430393490183815168)]} 0 2
[junit4:junit4] 2> 21489 T64 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[53 (1430393490183815168)]} 0 7
[junit4:junit4] 2> 21490 T94 C3 P44754 /update {version=2&wt=javabin} {add=[53]} 0 11
[junit4:junit4] 2> 21501 T92 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[54 (1430393490197446656)]} 0 2
[junit4:junit4] 2> 21501 T80 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[54 (1430393490197446656)]} 0 2
[junit4:junit4] 2> 21502 T67 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[54 (1430393490197446656)]} 0 7
[junit4:junit4] 2> 21503 T95 C3 P44754 /update {version=2&wt=javabin} {add=[54]} 0 11
[junit4:junit4] 2> 21522 T79 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[55 (1430393490219466752)]} 0 2
[junit4:junit4] 2> 21522 T94 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[55 (1430393490219466752)]} 0 2
[junit4:junit4] 2> 21523 T66 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[55 (1430393490219466752)]} 0 15
[junit4:junit4] 2> 21524 T93 C3 P44754 /update {version=2&wt=javabin} {add=[55]} 0 18
[junit4:junit4] 2> 21535 T78 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[56 (1430393490233098240)]} 0 2
[junit4:junit4] 2> 21536 T95 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[56 (1430393490233098240)]} 0 3
[junit4:junit4] 2> 21537 T65 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[56 (1430393490233098240)]} 0 8
[junit4:junit4] 2> 21538 T92 C3 P44754 /update {version=2&wt=javabin} {add=[56]} 0 11
[junit4:junit4] 2> 21553 T93 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[57 (1430393490251972608)]} 0 2
[junit4:junit4] 2> 21553 T81 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[57 (1430393490251972608)]} 0 2
[junit4:junit4] 2> 21554 T64 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[57 (1430393490251972608)]} 0 8
[junit4:junit4] 2> 21555 T94 C3 P44754 /update {version=2&wt=javabin} {add=[57]} 0 13
[junit4:junit4] 2> 21568 T92 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[58 (1430393490267701248)]} 0 2
[junit4:junit4] 2> 21568 T80 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[58 (1430393490267701248)]} 0 2
[junit4:junit4] 2> 21569 T67 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[58 (1430393490267701248)]} 0 8
[junit4:junit4] 2> 21570 T95 C3 P44754 /update {version=2&wt=javabin} {add=[58]} 0 12
[junit4:junit4] 2> 21581 T94 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[59 (1430393490281332736)]} 0 2
[junit4:junit4] 2> 21582 T79 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[59 (1430393490281332736)]} 0 3
[junit4:junit4] 2> 21583 T66 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[59 (1430393490281332736)]} 0 8
[junit4:junit4] 2> 21583 T93 C3 P44754 /update {version=2&wt=javabin} {add=[59]} 0 10
[junit4:junit4] 2> 21594 T78 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[60 (1430393490296012800)]} 0 2
[junit4:junit4] 2> 21595 T95 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[60 (1430393490296012800)]} 0 2
[junit4:junit4] 2> 21596 T65 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[60 (1430393490296012800)]} 0 8
[junit4:junit4] 2> 21597 T92 C3 P44754 /update {version=2&wt=javabin} {add=[60]} 0 11
[junit4:junit4] 2> 21607 T95 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[61 (1430393490309644288)]} 0 1
[junit4:junit4] 2> 21607 T81 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[61 (1430393490309644288)]} 0 1
[junit4:junit4] 2> 21608 T64 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[61 (1430393490309644288)]} 0 6
[junit4:junit4] 2> 21609 T94 C3 P44754 /update {version=2&wt=javabin} {add=[61]} 0 10
[junit4:junit4] 2> 21620 T80 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[62 (1430393490323275776)]} 0 2
[junit4:junit4] 2> 21620 T93 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[62 (1430393490323275776)]} 0 2
[junit4:junit4] 2> 21621 T67 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[62 (1430393490323275776)]} 0 7
[junit4:junit4] 2> 21622 T92 C3 P44754 /update {version=2&wt=javabin} {add=[62]} 0 10
[junit4:junit4] 2> 21632 T94 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[63 (1430393490335858688)]} 0 1
[junit4:junit4] 2> 21632 T78 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[63 (1430393490335858688)]} 0 1
[junit4:junit4] 2> 21634 T66 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[63 (1430393490335858688)]} 0 7
[junit4:junit4] 2> 21634 T95 C3 P44754 /update {version=2&wt=javabin} {add=[63]} 0 9
[junit4:junit4] 2> 21645 T81 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[64 (1430393490349490176)]} 0 2
[junit4:junit4] 2> 21645 T92 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[64 (1430393490349490176)]} 0 2
[junit4:junit4] 2> 21646 T65 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[64 (1430393490349490176)]} 0 7
[junit4:junit4] 2> 21647 T93 C3 P44754 /update {version=2&wt=javabin} {add=[64]} 0 10
[junit4:junit4] 2> 21657 T79 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[65 (1430393490362073088)]} 0 2
[junit4:junit4] 2> 21657 T95 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[65 (1430393490362073088)]} 0 2
[junit4:junit4] 2> 21659 T64 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[65 (1430393490362073088)]} 0 7
[junit4:junit4] 2> 21660 T94 C3 P44754 /update {version=2&wt=javabin} {add=[65]} 0 11
[junit4:junit4] 2> 21671 T80 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[66 (1430393490376753152)]} 0 1
[junit4:junit4] 2> 21671 T93 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[66 (1430393490376753152)]} 0 1
[junit4:junit4] 2> 21672 T67 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[66 (1430393490376753152)]} 0 7
[junit4:junit4] 2> 21673 T92 C3 P44754 /update {version=2&wt=javabin} {add=[66]} 0 10
[junit4:junit4] 2> 21684 T94 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[67 (1430393490390384640)]} 0 1
[junit4:junit4] 2> 21684 T78 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[67 (1430393490390384640)]} 0 1
[junit4:junit4] 2> 21685 T66 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[67 (1430393490390384640)]} 0 6
[junit4:junit4] 2> 21686 T95 C3 P44754 /update {version=2&wt=javabin} {add=[67]} 0 10
[junit4:junit4] 2> 21697 T92 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[68 (1430393490404016128)]} 0 2
[junit4:junit4] 2> 21697 T81 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[68 (1430393490404016128)]} 0 2
[junit4:junit4] 2> 21698 T65 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[68 (1430393490404016128)]} 0 7
[junit4:junit4] 2> 21699 T93 C3 P44754 /update {version=2&wt=javabin} {add=[68]} 0 10
[junit4:junit4] 2> 21711 T79 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[69 (1430393490417647616)]} 0 2
[junit4:junit4] 2> 21711 T95 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[69 (1430393490417647616)]} 0 2
[junit4:junit4] 2> 21712 T64 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[69 (1430393490417647616)]} 0 8
[junit4:junit4] 2> 21713 T94 C3 P44754 /update {version=2&wt=javabin} {add=[69]} 0 11
[junit4:junit4] 2> 21723 T93 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[70 (1430393490431279104)]} 0 1
[junit4:junit4] 2> 21724 T80 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[70 (1430393490431279104)]} 0 2
[junit4:junit4] 2> 21725 T67 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[70 (1430393490431279104)]} 0 7
[junit4:junit4] 2> 21725 T92 C3 P44754 /update {version=2&wt=javabin} {add=[70]} 0 10
[junit4:junit4] 2> 21736 T94 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[71 (1430393490444910592)]} 0 2
[junit4:junit4] 2> 21736 T78 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[71 (1430393490444910592)]} 0 2
[junit4:junit4] 2> 21737 T66 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[71 (1430393490444910592)]} 0 7
[junit4:junit4] 2> 21738 T95 C3 P44754 /update {version=2&wt=javabin} {add=[71]} 0 10
[junit4:junit4] 2> 21748 T92 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[72 (1430393490457493504)]} 0 2
[junit4:junit4] 2> 21748 T81 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[72 (1430393490457493504)]} 0 1
[junit4:junit4] 2> 21749 T65 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[72 (1430393490457493504)]} 0 6
[junit4:junit4] 2> 21750 T93 C3 P44754 /update {version=2&wt=javabin} {add=[72]} 0 9
[junit4:junit4] 2> 21761 T95 C3 P44754 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[73 (1430393490470076416)]} 0 2
[junit4:junit4] 2> 21761 T79 C1 P45516 /update {update.distrib=FROMLEADER&version=2&distrib.from=http://127.0.0.1:43934/unloadcollection1/&wt=javabin} {add=[73 (1430393490470076416)]} 0 2
[junit4:junit4] 2> 21762 T64 C2 P43934 /update {update.distrib=TOLEADER&version=2&distrib.from=http://127.0.0.1:44754/unloadcollection3/&wt=javabin} {add=[73
[...truncated too long message...]
junit4] 1> "core":"collection1",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:42989"}}}},
[junit4:junit4] 1> "router":"compositeId"}}
[junit4:junit4] 1> /solr/overseer (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1> /solr/overseer/collection-queue-work (0)
[junit4:junit4] 1> /solr/overseer/queue (0)
[junit4:junit4] 1> /solr/overseer/queue-work (0)
[junit4:junit4] 1> /zookeeper (1)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1>
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=BasicDistributedZk2Test -Dtests.method=testDistribSearch -Dtests.seed=12AF573644669E6B -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=en_SG -Dtests.timezone=Asia/Istanbul -Dtests.file.encoding=UTF-8
[junit4:junit4] ERROR 32.7s J1 | BasicDistributedZk2Test.testDistribSearch <<<
[junit4:junit4] > Throwable #1: org.apache.solr.common.SolrException: Server at http://127.0.0.1:34480/onenodecollectioncore returned non ok status:404, message:Can not find: /onenodecollectioncore/update
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([12AF573644669E6B:9349D92E3339FE57]:0)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:375)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181)
[junit4:junit4] > at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
[junit4:junit4] > at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:116)
[junit4:junit4] > at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:102)
[junit4:junit4] > at org.apache.solr.cloud.BasicDistributedZk2Test.testNodeWithoutCollectionForwarding(BasicDistributedZk2Test.java:197)
[junit4:junit4] > at org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:89)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
[junit4:junit4] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4:junit4] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[junit4:junit4] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4:junit4] > at java.lang.reflect.Method.invoke(Method.java:487)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4] > at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4:junit4] > at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4:junit4] > at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4] > at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
[junit4:junit4] > at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4] > at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4] > at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4] > at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4] > at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4] > at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4] > at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2> 32737 T234 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 32740 T233 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 2 leaked thread(s).
[junit4:junit4] 2> 33461 T237 oazs.SessionTrackerImpl.run SessionTrackerImpl exited loop!
[junit4:junit4] 2> 33896 T325 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 33897 T325 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 33897 T325 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 33897 T325 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> NOTE: test params are: codec=Lucene40, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=en_SG, timezone=Asia/Istanbul
[junit4:junit4] 2> NOTE: Linux 3.2.0-39-generic i386/Oracle Corporation 1.8.0-ea (32-bit)/cpus=8,threads=2,free=24255272,total=94502912
[junit4:junit4] 2> NOTE: All tests run in this JVM: [HighlighterTest, OpenExchangeRatesOrgProviderTest, EchoParamsTest, ShardRoutingCustomTest, TestCollationField, DirectUpdateHandlerTest, FileUtilsTest, TestStressLucene, TimeZoneUtilsTest, DisMaxRequestHandlerTest, TestHashPartitioner, XmlUpdateRequestHandlerTest, SpellCheckComponentTest, TestSuggestSpellingConverter, DefaultValueUpdateProcessorTest, TestWordDelimiterFilterFactory, ZkNodePropsTest, HardAutoCommitTest, TestFiltering, NumericFieldsTest, RequestHandlersTest, NoCacheHeaderTest, TestStressReorder, DOMUtilTest, TestElisionMultitermQuery, SpellPossibilityIteratorTest, BasicDistributedZk2Test]
[junit4:junit4] Completed on J1 in 34.03s, 1 test, 1 error <<< FAILURES!
[...truncated 758 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:381: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:361: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:183: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:449: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1212: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:876: There were test failures: 273 suites, 1149 tests, 1 error, 1 failure, 13 ignored (7 assumptions)
Total time: 39 minutes 32 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Description set: Java: 32bit/jdk1.8.0-ea-b82 -server -XX:+UseConcMarkSweepGC
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b82) - Build #
4805 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/4805/
Java: 32bit/jdk1.8.0-ea-b82 -server -XX:+UseG1GC -XX:MarkStackSize=256K
1 tests failed.
FAILED: org.apache.solr.cloud.BasicDistributedZk2Test.testDistribSearch
Error Message:
Server at http://127.0.0.1:52380/onenodecollectioncore returned non ok status:404, message:Can not find: /onenodecollectioncore/update
Stack Trace:
org.apache.solr.common.SolrException: Server at http://127.0.0.1:52380/onenodecollectioncore returned non ok status:404, message:Can not find: /onenodecollectioncore/update
at __randomizedtesting.SeedInfo.seed([995C80BF3A3661E9:18BA0EA74D6901D5]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:375)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181)
at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:116)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:102)
at org.apache.solr.cloud.BasicDistributedZk2Test.testNodeWithoutCollectionForwarding(BasicDistributedZk2Test.java:197)
at org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:89)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:487)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Build Log:
[...truncated 9326 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZk2Test
[junit4:junit4] 2> 0 T2185 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4] 2> 3 T2185 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1364124750444
[junit4:junit4] 2> 4 T2185 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 4 T2186 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 5 T2186 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4] 2> 5 T2186 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4] 2> 5 T2186 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4] 2> 5 T2186 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4] 2> 6 T2186 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1364124750443/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4] 2> 104 T2185 oasc.ZkTestServer.run start zk server on port:54134
[junit4:junit4] 2> 105 T2185 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54134 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@9bef6b
[junit4:junit4] 2> 106 T2185 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 107 T2191 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54134. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 107 T2191 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54134, initiating session
[junit4:junit4] 2> 107 T2187 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:41172
[junit4:junit4] 2> 108 T2187 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:41172
[junit4:junit4] 2> 108 T2189 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4] 2> 162 T2189 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9c2bd26f0000 with negotiated timeout 10000 for client /127.0.0.1:41172
[junit4:junit4] 2> 163 T2191 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54134, sessionid = 0x13d9c2bd26f0000, negotiated timeout = 10000
[junit4:junit4] 2> 163 T2192 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@9bef6b name:ZooKeeperConnection Watcher:127.0.0.1:54134 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 164 T2185 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 164 T2185 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 170 T2190 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d9c2bd26f0000
[junit4:junit4] 2> 171 T2187 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:41172 which had sessionid 0x13d9c2bd26f0000
[junit4:junit4] 2> 171 T2192 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 171 T2185 oaz.ZooKeeper.close Session: 0x13d9c2bd26f0000 closed
[junit4:junit4] 2> 172 T2185 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54134/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@5c1988
[junit4:junit4] 2> 174 T2193 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54134. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 174 T2185 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 174 T2193 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54134, initiating session
[junit4:junit4] 2> 175 T2187 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:41173
[junit4:junit4] 2> 175 T2187 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:41173
[junit4:junit4] 2> 176 T2189 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9c2bd26f0001 with negotiated timeout 10000 for client /127.0.0.1:41173
[junit4:junit4] 2> 176 T2193 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54134, sessionid = 0x13d9c2bd26f0001, negotiated timeout = 10000
[junit4:junit4] 2> 177 T2194 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5c1988 name:ZooKeeperConnection Watcher:127.0.0.1:54134/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 177 T2185 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 178 T2185 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 184 T2185 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 188 T2185 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 192 T2185 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 197 T2185 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 198 T2185 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 205 T2185 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 206 T2185 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 212 T2185 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 213 T2185 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 218 T2185 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 218 T2185 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 223 T2185 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 224 T2185 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 228 T2185 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 229 T2185 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 234 T2185 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 235 T2185 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 240 T2185 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 241 T2185 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 245 T2185 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 246 T2185 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 251 T2190 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d9c2bd26f0001
[junit4:junit4] 2> 252 T2194 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 252 T2187 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:41173 which had sessionid 0x13d9c2bd26f0001
[junit4:junit4] 2> 252 T2185 oaz.ZooKeeper.close Session: 0x13d9c2bd26f0001 closed
[junit4:junit4] 2> 325 T2185 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 332 T2185 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50017
[junit4:junit4] 2> 333 T2185 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 333 T2185 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 333 T2185 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1364124750693
[junit4:junit4] 2> 334 T2185 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1364124750693/solr.xml
[junit4:junit4] 2> 334 T2185 oasc.CoreContainer.<init> New CoreContainer 20301957
[junit4:junit4] 2> 334 T2185 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1364124750693/'
[junit4:junit4] 2> 335 T2185 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1364124750693/'
[junit4:junit4] 2> 352 T2185 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 352 T2185 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 353 T2185 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 353 T2185 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 353 T2185 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 354 T2185 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 354 T2185 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 354 T2185 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 355 T2185 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 355 T2185 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 360 T2185 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 369 T2185 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:54134/solr
[junit4:junit4] 2> 370 T2185 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 370 T2185 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54134 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@1871d3e
[junit4:junit4] 2> 371 T2204 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54134. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 371 T2185 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 372 T2204 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54134, initiating session
[junit4:junit4] 2> 372 T2187 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:41174
[junit4:junit4] 2> 372 T2187 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:41174
[junit4:junit4] 2> 373 T2189 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9c2bd26f0002 with negotiated timeout 20000 for client /127.0.0.1:41174
[junit4:junit4] 2> 373 T2204 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54134, sessionid = 0x13d9c2bd26f0002, negotiated timeout = 20000
[junit4:junit4] 2> 374 T2205 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1871d3e name:ZooKeeperConnection Watcher:127.0.0.1:54134 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 374 T2185 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 375 T2190 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d9c2bd26f0002
[junit4:junit4] 2> 376 T2205 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 376 T2187 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:41174 which had sessionid 0x13d9c2bd26f0002
[junit4:junit4] 2> 376 T2185 oaz.ZooKeeper.close Session: 0x13d9c2bd26f0002 closed
[junit4:junit4] 2> 377 T2185 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 379 T2185 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54134/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@1c9e6a6
[junit4:junit4] 2> 380 T2206 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54134. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 380 T2185 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 380 T2206 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54134, initiating session
[junit4:junit4] 2> 380 T2187 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:41175
[junit4:junit4] 2> 381 T2187 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:41175
[junit4:junit4] 2> 381 T2189 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9c2bd26f0003 with negotiated timeout 20000 for client /127.0.0.1:41175
[junit4:junit4] 2> 381 T2206 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54134, sessionid = 0x13d9c2bd26f0003, negotiated timeout = 20000
[junit4:junit4] 2> 382 T2207 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c9e6a6 name:ZooKeeperConnection Watcher:127.0.0.1:54134/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 382 T2185 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 384 T2185 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 387 T2190 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c2bd26f0003 type:create cxid:0x7 zxid:0x1b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 388 T2185 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 393 T2185 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 396 T2185 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50017_
[junit4:junit4] 2> 397 T2190 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c2bd26f0003 type:delete cxid:0x12 zxid:0x1e txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:50017_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:50017_
[junit4:junit4] 2> 398 T2185 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50017_
[junit4:junit4] 2> 403 T2185 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 410 T2190 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c2bd26f0003 type:delete cxid:0x20 zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4] 2> 411 T2185 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 415 T2185 oasc.Overseer.start Overseer (id=89399279645294595-127.0.0.1:50017_-n_0000000000) starting
[junit4:junit4] 2> 416 T2190 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c2bd26f0003 type:create cxid:0x25 zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 418 T2190 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c2bd26f0003 type:create cxid:0x26 zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 419 T2190 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c2bd26f0003 type:create cxid:0x28 zxid:0x27 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 421 T2185 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 425 T2190 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c2bd26f0003 type:create cxid:0x2e zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 427 T2209 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 428 T2185 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 432 T2185 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 435 T2185 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 440 T2208 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 443 T2210 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1364124750693/collection1
[junit4:junit4] 2> 443 T2210 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 444 T2210 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 445 T2210 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 447 T2210 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1364124750693/collection1/'
[junit4:junit4] 2> 448 T2210 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1364124750693/collection1/lib/README' to classloader
[junit4:junit4] 2> 448 T2210 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1364124750693/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 485 T2210 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4] 2> 512 T2210 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 513 T2210 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 517 T2210 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 812 T2210 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 817 T2210 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 819 T2210 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 837 T2210 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 841 T2210 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 844 T2210 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 846 T2210 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 846 T2210 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 846 T2210 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 848 T2210 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 848 T2210 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 849 T2210 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 849 T2210 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1364124750693/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1364124750443/control/data/
[junit4:junit4] 2> 849 T2210 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@11b579d
[junit4:junit4] 2> 850 T2210 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 851 T2210 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1364124750443/control/data
[junit4:junit4] 2> 851 T2210 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1364124750443/control/data/index/
[junit4:junit4] 2> 851 T2210 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1364124750443/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 852 T2210 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1364124750443/control/data/index
[junit4:junit4] 2> 854 T2210 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@b26544 lockFactory=org.apache.lucene.store.NativeFSLockFactory@29cc3b),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 854 T2210 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 856 T2210 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 857 T2210 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 857 T2210 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 858 T2210 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 858 T2210 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 859 T2210 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 859 T2210 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 860 T2210 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 860 T2210 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 864 T2210 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 867 T2210 oass.SolrIndexSearcher.<init> Opening Searcher@1ed2e99 main
[junit4:junit4] 2> 868 T2210 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1364124750443/control/data/tlog
[junit4:junit4] 2> 868 T2210 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 869 T2210 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 873 T2211 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1ed2e99 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 874 T2210 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 874 T2210 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1946 T2208 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1947 T2208 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "node_name":"127.0.0.1:50017_",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50017",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "roles":null}
[junit4:junit4] 2> 1948 T2208 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 1948 T2208 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 1955 T2207 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 2876 T2210 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 2877 T2210 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:50017 collection:control_collection shard:shard1
[junit4:junit4] 2> 2878 T2210 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 2890 T2190 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c2bd26f0003 type:delete cxid:0x82 zxid:0x35 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4] 2> 2891 T2210 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 2892 T2190 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c2bd26f0003 type:create cxid:0x83 zxid:0x36 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 2896 T2210 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 2897 T2210 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 2897 T2210 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:50017/collection1/
[junit4:junit4] 2> 2897 T2210 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 2898 T2210 oasc.SyncStrategy.syncToMe http://127.0.0.1:50017/collection1/ has no replicas
[junit4:junit4] 2> 2898 T2210 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:50017/collection1/
[junit4:junit4] 2> 2898 T2210 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 2904 T2190 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c2bd26f0003 type:create cxid:0x8e zxid:0x3a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 3463 T2208 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 3473 T2207 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 3510 T2210 oasc.ZkController.register We are http://127.0.0.1:50017/collection1/ and leader is http://127.0.0.1:50017/collection1/
[junit4:junit4] 2> 3510 T2210 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:50017
[junit4:junit4] 2> 3510 T2210 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 3511 T2210 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 3511 T2210 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 3513 T2210 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 3515 T2185 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 3516 T2185 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 3516 T2185 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 3522 T2185 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 3523 T2185 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54134/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@1e25de4
[junit4:junit4] 2> 3525 T2213 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54134. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 3525 T2185 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 3525 T2213 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54134, initiating session
[junit4:junit4] 2> 3526 T2187 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:41176
[junit4:junit4] 2> 3526 T2187 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:41176
[junit4:junit4] 2> 3527 T2189 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9c2bd26f0004 with negotiated timeout 10000 for client /127.0.0.1:41176
[junit4:junit4] 2> 3528 T2213 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54134, sessionid = 0x13d9c2bd26f0004, negotiated timeout = 10000
[junit4:junit4] 2> 3528 T2214 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1e25de4 name:ZooKeeperConnection Watcher:127.0.0.1:54134/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 3528 T2185 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 3531 T2185 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 3534 T2185 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 3603 T2185 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 3606 T2185 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:41035
[junit4:junit4] 2> 3607 T2185 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 3608 T2185 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 3609 T2185 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1364124753975
[junit4:junit4] 2> 3609 T2185 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1364124753975/solr.xml
[junit4:junit4] 2> 3610 T2185 oasc.CoreContainer.<init> New CoreContainer 31263381
[junit4:junit4] 2> 3611 T2185 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1364124753975/'
[junit4:junit4] 2> 3612 T2185 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1364124753975/'
[junit4:junit4] 2> 3637 T2185 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 3638 T2185 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 3638 T2185 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 3638 T2185 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 3639 T2185 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 3639 T2185 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 3639 T2185 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 3640 T2185 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 3640 T2185 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 3640 T2185 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 3646 T2185 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 3655 T2185 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:54134/solr
[junit4:junit4] 2> 3655 T2185 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 3655 T2185 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54134 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@4a3310
[junit4:junit4] 2> 3657 T2224 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54134. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 3656 T2185 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 3657 T2224 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54134, initiating session
[junit4:junit4] 2> 3658 T2187 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:41177
[junit4:junit4] 2> 3658 T2187 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:41177
[junit4:junit4] 2> 3659 T2189 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9c2bd26f0005 with negotiated timeout 20000 for client /127.0.0.1:41177
[junit4:junit4] 2> 3659 T2224 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54134, sessionid = 0x13d9c2bd26f0005, negotiated timeout = 20000
[junit4:junit4] 2> 3659 T2225 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4a3310 name:ZooKeeperConnection Watcher:127.0.0.1:54134 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 3660 T2185 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 3661 T2190 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d9c2bd26f0005
[junit4:junit4] 2> 3662 T2225 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 3662 T2187 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:41177 which had sessionid 0x13d9c2bd26f0005
[junit4:junit4] 2> 3662 T2185 oaz.ZooKeeper.close Session: 0x13d9c2bd26f0005 closed
[junit4:junit4] 2> 3663 T2185 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 3665 T2185 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54134/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@343421
[junit4:junit4] 2> 3666 T2226 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54134. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 3666 T2185 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 3666 T2226 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54134, initiating session
[junit4:junit4] 2> 3666 T2187 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:41178
[junit4:junit4] 2> 3667 T2187 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:41178
[junit4:junit4] 2> 3668 T2189 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9c2bd26f0006 with negotiated timeout 20000 for client /127.0.0.1:41178
[junit4:junit4] 2> 3668 T2226 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54134, sessionid = 0x13d9c2bd26f0006, negotiated timeout = 20000
[junit4:junit4] 2> 3668 T2227 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@343421 name:ZooKeeperConnection Watcher:127.0.0.1:54134/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 3669 T2185 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 3670 T2190 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c2bd26f0006 type:create cxid:0x1 zxid:0x48 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 3671 T2190 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c2bd26f0006 type:create cxid:0x3 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 3674 T2185 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 4678 T2185 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41035_
[junit4:junit4] 2> 4680 T2190 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c2bd26f0006 type:delete cxid:0xd zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:41035_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:41035_
[junit4:junit4] 2> 4681 T2185 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:41035_
[junit4:junit4] 2> 4686 T2207 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 4687 T2227 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 4687 T2214 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 4689 T2207 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 4696 T2228 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1364124753975/collection1
[junit4:junit4] 2> 4696 T2228 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 4698 T2228 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 4698 T2228 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 4700 T2228 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1364124753975/collection1/'
[junit4:junit4] 2> 4701 T2228 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1364124753975/collection1/lib/README' to classloader
[junit4:junit4] 2> 4701 T2228 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1364124753975/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 4738 T2228 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4] 2> 4794 T2228 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 4796 T2228 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 4799 T2228 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 4981 T2208 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 4982 T2208 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "node_name":"127.0.0.1:50017_",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50017",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "roles":null}
[junit4:junit4] 2> 4997 T2207 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 4997 T2227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 4997 T2214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 5099 T2228 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 5104 T2228 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 5106 T2228 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 5124 T2228 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 5127 T2228 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 5133 T2228 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 5134 T2228 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 5135 T2228 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 5135 T2228 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 5137 T2228 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 5138 T2228 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 5138 T2228 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 5138 T2228 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1364124753975/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1364124750443/jetty1/
[junit4:junit4] 2> 5139 T2228 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@11b579d
[junit4:junit4] 2> 5139 T2228 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 5140 T2228 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1364124750443/jetty1
[junit4:junit4] 2> 5141 T2228 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1364124750443/jetty1/index/
[junit4:junit4] 2> 5141 T2228 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1364124750443/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 5141 T2228 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1364124750443/jetty1/index
[junit4:junit4] 2> 5144 T2228 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1fb3191 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1ba4243),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 5144 T2228 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 5147 T2228 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 5148 T2228 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 5148 T2228 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 5149 T2228 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 5150 T2228 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 5150 T2228 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 5151 T2228 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 5152 T2228 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 5152 T2228 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 5157 T2228 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 5161 T2228 oass.SolrIndexSearcher.<init> Opening Searcher@14de489 main
[junit4:junit4] 2> 5161 T2228 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1364124750443/jetty1/tlog
[junit4:junit4] 2> 5162 T2228 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 5162 T2228 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 5169 T2229 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@14de489 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 5171 T2228 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 5172 T2228 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 6503 T2208 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 6505 T2208 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "node_name":"127.0.0.1:41035_",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:41035",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "roles":null}
[junit4:junit4] 2> 6505 T2208 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4] 2> 6505 T2208 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 6513 T2207 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 6513 T2214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 6513 T2227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 7174 T2228 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 7174 T2228 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:41035 collection:collection1 shard:shard1
[junit4:junit4] 2> 7176 T2228 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 7186 T2190 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c2bd26f0006 type:delete cxid:0x4a zxid:0x5a txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4] 2> 7187 T2228 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 7188 T2190 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c2bd26f0006 type:create cxid:0x4b zxid:0x5b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 7191 T2228 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 7191 T2228 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 7192 T2228 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:41035/collection1/
[junit4:junit4] 2> 7192 T2228 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 7192 T2228 oasc.SyncStrategy.syncToMe http://127.0.0.1:41035/collection1/ has no replicas
[junit4:junit4] 2> 7193 T2228 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:41035/collection1/
[junit4:junit4] 2> 7193 T2228 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 7200 T2190 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c2bd26f0006 type:create cxid:0x56 zxid:0x5f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 8021 T2208 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 8035 T2214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 8036 T2207 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 8035 T2227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 8056 T2228 oasc.ZkController.register We are http://127.0.0.1:41035/collection1/ and leader is http://127.0.0.1:41035/collection1/
[junit4:junit4] 2> 8057 T2228 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:41035
[junit4:junit4] 2> 8057 T2228 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 8057 T2228 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 8058 T2228 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 8060 T2228 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 8062 T2185 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 8063 T2185 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 8064 T2185 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 8130 T2185 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 8133 T2185 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:52380
[junit4:junit4] 2> 8134 T2185 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 8134 T2185 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 8135 T2185 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1364124758509
[junit4:junit4] 2> 8135 T2185 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1364124758509/solr.xml
[junit4:junit4] 2> 8136 T2185 oasc.CoreContainer.<init> New CoreContainer 2286710
[junit4:junit4] 2> 8137 T2185 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1364124758509/'
[junit4:junit4] 2> 8138 T2185 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1364124758509/'
[junit4:junit4] 2> 8162 T2185 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 8162 T2185 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 8163 T2185 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 8163 T2185 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 8163 T2185 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 8164 T2185 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 8164 T2185 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 8164 T2185 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 8165 T2185 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 8165 T2185 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 8170 T2185 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 8179 T2185 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:54134/solr
[junit4:junit4] 2> 8179 T2185 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 8180 T2185 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54134 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@6292cb
[junit4:junit4] 2> 8181 T2240 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54134. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 8181 T2185 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 8181 T2240 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54134, initiating session
[junit4:junit4] 2> 8182 T2187 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:41180
[junit4:junit4] 2> 8182 T2187 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:41180
[junit4:junit4] 2> 8183 T2189 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9c2bd26f0007 with negotiated timeout 20000 for client /127.0.0.1:41180
[junit4:junit4] 2> 8183 T2240 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54134, sessionid = 0x13d9c2bd26f0007, negotiated timeout = 20000
[junit4:junit4] 2> 8183 T2241 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6292cb name:ZooKeeperConnection Watcher:127.0.0.1:54134 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 8183 T2185 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 8185 T2190 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d9c2bd26f0007
[junit4:junit4] 2> 8186 T2241 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 8186 T2187 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:41180 which had sessionid 0x13d9c2bd26f0007
[junit4:junit4] 2> 8186 T2185 oaz.ZooKeeper.close Session: 0x13d9c2bd26f0007 closed
[junit4:junit4] 2> 8187 T2185 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 8189 T2185 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54134/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@1eaa2e2
[junit4:junit4] 2> 8190 T2242 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54134. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 8190 T2185 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 8190 T2242 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54134, initiating session
[junit4:junit4] 2> 8190 T2187 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:41181
[junit4:junit4] 2> 8191 T2187 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:41181
[junit4:junit4] 2> 8192 T2189 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9c2bd26f0008 with negotiated timeout 20000 for client /127.0.0.1:41181
[junit4:junit4] 2> 8192 T2242 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54134, sessionid = 0x13d9c2bd26f0008, negotiated timeout = 20000
[junit4:junit4] 2> 8192 T2243 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1eaa2e2 name:ZooKeeperConnection Watcher:127.0.0.1:54134/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 8193 T2185 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 8194 T2190 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c2bd26f0008 type:create cxid:0x1 zxid:0x6c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 8195 T2190 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c2bd26f0008 type:create cxid:0x3 zxid:0x6d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 8199 T2185 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 9203 T2185 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52380_
[junit4:junit4] 2> 9205 T2190 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c2bd26f0008 type:delete cxid:0xd zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:52380_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:52380_
[junit4:junit4] 2> 9206 T2185 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52380_
[junit4:junit4] 2> 9209 T2214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 9209 T2227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 9210 T2207 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 9213 T2243 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9215 T2227 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9215 T2214 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9216 T2207 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9222 T2244 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1364124758509/collection1
[junit4:junit4] 2> 9222 T2244 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 9223 T2244 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 9224 T2244 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 9226 T2244 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1364124758509/collection1/'
[junit4:junit4] 2> 9226 T2244 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1364124758509/collection1/lib/README' to classloader
[junit4:junit4] 2> 9227 T2244 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1364124758509/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 9260 T2244 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4] 2> 9287 T2244 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 9288 T2244 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 9292 T2244 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 9553 T2208 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 9554 T2208 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "node_name":"127.0.0.1:41035_",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:41035",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "roles":null}
[junit4:junit4] 2> 9557 T2227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9557 T2214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9558 T2207 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9557 T2243 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9602 T2244 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 9607 T2244 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 9609 T2244 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 9628 T2244 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 9631 T2244 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 9637 T2244 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 9639 T2244 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 9640 T2244 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 9640 T2244 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 9642 T2244 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 9642 T2244 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 9643 T2244 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 9643 T2244 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1364124758509/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1364124750443/jetty2/
[junit4:junit4] 2> 9644 T2244 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@11b579d
[junit4:junit4] 2> 9644 T2244 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 9645 T2244 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1364124750443/jetty2
[junit4:junit4] 2> 9646 T2244 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1364124750443/jetty2/index/
[junit4:junit4] 2> 9646 T2244 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1364124750443/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 9646 T2244 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1364124750443/jetty2/index
[junit4:junit4] 2> 9649 T2244 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@a65a4a lockFactory=org.apache.lucene.store.NativeFSLockFactory@17d94e0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 9649 T2244 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 9652 T2244 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 9652 T2244 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 9653 T2244 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 9654 T2244 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 9655 T2244 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 9655 T2244 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 9656 T2244 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 9657 T2244 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 9657 T2244 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 9662 T2244 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 9666 T2244 oass.SolrIndexSearcher.<init> Opening Searcher@2770d1 main
[junit4:junit4] 2> 9666 T2244 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1364124750443/jetty2/tlog
[junit4:junit4] 2> 9667 T2244 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 9667 T2244 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 9670 T2245 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2770d1 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 9671 T2244 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 9672 T2244 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 11064 T2208 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 11065 T2208 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "node_name":"127.0.0.1:52380_",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:52380",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "roles":null}
[junit4:junit4] 2> 11066 T2208 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 11066 T2208 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4] 2> 11073 T2243 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 11073 T2227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 11073 T2214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 11073 T2207 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 11674 T2244 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 11674 T2244 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:52380 collection:collection1 shard:shard2
[junit4:junit4] 2> 11675 T2244 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 11681 T2190 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c2bd26f0008 type:delete cxid:0x49 zxid:0x7d txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
[junit4:junit4] 2> 11682 T2244 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 11683 T2190 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c2bd26f0008 type:create cxid:0x4a zxid:0x7e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 11685 T2244 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 11685 T2244 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 11685 T2244 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:52380/collection1/
[junit4:junit4] 2> 11686 T2244 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 11686 T2244 oasc.SyncStrategy.syncToMe http://127.0.0.1:52380/collection1/ has no replicas
[junit4:junit4] 2> 11686 T2244 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:52380/collection1/
[junit4:junit4] 2> 11686 T2244 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 11689 T2190 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c2bd26f0008 type:create cxid:0x54 zxid:0x81 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 12579 T2208 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 12590 T2227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 12590 T2214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 12590 T2207 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 12590 T2243 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 12594 T2244 oasc.ZkController.register We are http://127.0.0.1:52380/collection1/ and leader is http://127.0.0.1:52380/collection1/
[junit4:junit4] 2> 12595 T2244 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:52380
[junit4:junit4] 2> 12595 T2244 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 12595 T2244 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 12596 T2244 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 12598 T2244 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 12600 T2185 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 12600 T2185 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 12601 T2185 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 12673 T2185 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 12675 T2185 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:38174
[junit4:junit4] 2> 12676 T2185 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 12677 T2185 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 12677 T2185 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1364124763047
[junit4:junit4] 2> 12678 T2185 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1364124763047/solr.xml
[junit4:junit4] 2> 12678 T2185 oasc.CoreContainer.<init> New CoreContainer 8656520
[junit4:junit4] 2> 12678 T2185 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1364124763047/'
[junit4:junit4] 2> 12679 T2185 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1364124763047/'
[junit4:junit4] 2> 12699 T2185 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 12700 T2185 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 12700 T2185 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 12701 T2185 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 12701 T2185 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 12701 T2185 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 12702 T2185 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 12702 T2185 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 12702 T2185 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 12703 T2185 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 12708 T2185 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 12718 T2185 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:54134/solr
[junit4:junit4] 2> 12718 T2185 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 12719 T2185 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54134 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@cb7683
[junit4:junit4] 2> 12720 T2256 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54134. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 12720 T2185 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 12721 T2256 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54134, initiating session
[junit4:junit4] 2> 12721 T2187 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:41188
[junit4:junit4] 2> 12721 T2187 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:41188
[junit4:junit4] 2> 12722 T2189 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9c2bd26f0009 with negotiated timeout 20000 for client /127.0.0.1:41188
[junit4:junit4] 2> 12722 T2256 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54134, sessionid = 0x13d9c2bd26f0009, negotiated timeout = 20000
[junit4:junit4] 2> 12722 T2257 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@cb7683 name:ZooKeeperConnection Watcher:127.0.0.1:54134 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 12723 T2185 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 12724 T2190 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d9c2bd26f0009
[junit4:junit4] 2> 12725 T2257 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 12725 T2187 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:41188 which had sessionid 0x13d9c2bd26f0009
[junit4:junit4] 2> 12725 T2185 oaz.ZooKeeper.close Session: 0x13d9c2bd26f0009 closed
[junit4:junit4] 2> 12726 T2185 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 12728 T2185 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54134/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@13e6d98
[junit4:junit4] 2> 12729 T2258 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54134. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 12729 T2185 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 12729 T2258 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54134, initiating session
[junit4:junit4] 2> 12729 T2187 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:41189
[junit4:junit4] 2> 12730 T2187 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:41189
[junit4:junit4] 2> 12731 T2189 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9c2bd26f000a with negotiated timeout 20000 for client /127.0.0.1:41189
[junit4:junit4] 2> 12731 T2258 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54134, sessionid = 0x13d9c2bd26f000a, negotiated timeout = 20000
[junit4:junit4] 2> 12731 T2259 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@13e6d98 name:ZooKeeperConnection Watcher:127.0.0.1:54134/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 12731 T2185 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 12732 T2190 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c2bd26f000a type:create cxid:0x1 zxid:0x8e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 12734 T2190 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c2bd26f000a type:create cxid:0x3 zxid:0x8f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 12735 T2185 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 13738 T2185 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38174_
[junit4:junit4] 2> 13739 T2190 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c2bd26f000a type:delete cxid:0xd zxid:0x90 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:38174_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:38174_
[junit4:junit4] 2> 13739 T2185 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:38174_
[junit4:junit4] 2> 13741 T2243 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 13742 T2214 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 13743 T2227 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 13743 T2214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 13743 T2259 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 13743 T2227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 13743 T2207 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 13743 T2207 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 13745 T2243 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 13751 T2260 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1364124763047/collection1
[junit4:junit4] 2> 13751 T2260 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 13752 T2260 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 13752 T2260 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 13753 T2260 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1364124763047/collection1/'
[junit4:junit4] 2> 13753 T2260 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1364124763047/collection1/lib/README' to classloader
[junit4:junit4] 2> 13754 T2260 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1364124763047/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 13779 T2260 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4] 2> 13809 T2260 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 13809 T2260 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 13813 T2260 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 14098 T2208 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 14098 T2208 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "core_node_name":"3",
[junit4:junit4] 2> "node_name":"127.0.0.1:52380_",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:52380",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "roles":null}
[junit4:junit4] 2> 14102 T2227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14102 T2259 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14102 T2214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14102 T2243 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14102 T2207 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14163 T2260 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 14169 T2260 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 14171 T2260 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 14183 T2260 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 14187 T2260 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 14191 T2260 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 14193 T2260 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 14193 T2260 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 14193 T2260 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 14195 T2260 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 14195 T2260 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 14195 T2260 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 14195 T2260 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1364124763047/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1364124750443/jetty3/
[junit4:junit4] 2> 14196 T2260 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@11b579d
[junit4:junit4] 2> 14196 T2260 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 14197 T2260 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1364124750443/jetty3
[junit4:junit4] 2> 14197 T2260 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1364124750443/jetty3/index/
[junit4:junit4] 2> 14197 T2260 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1364124750443/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 14197 T2260 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1364124750443/jetty3/index
[junit4:junit4] 2> 14198 T2260 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@182ecfa lockFactory=org.apache.lucene.store.NativeFSLockFactory@7795a9),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 14199 T2260 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 14200 T2260 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 14200 T2260 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 14201 T2260 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 14201 T2260 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 14202 T2260 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 14202 T2260 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 14202 T2260 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 14203 T2260 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 14203 T2260 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 14205 T2260 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 14207 T2260 oass.SolrIndexSearcher.<init> Opening Searcher@98de9d main
[junit4:junit4] 2> 14208 T2260 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1364124750443/jetty3/tlog
[junit4:junit4] 2> 14208 T2260 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 14208 T2260 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 14212 T2261 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@98de9d main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 14213 T2260 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 14213 T2260 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 15607 T2208 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 15608 T2208 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "node_name":"127.0.0.1:38174_",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:38174",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "roles":null}
[junit4:junit4] 2> 15608 T2208 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 15608 T2208 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 15612 T2259 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 15612 T2227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 15613 T2207 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 15613 T2214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 15613 T2243 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 16215 T2260 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 16215 T2260 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:38174 collection:collection1 shard:shard1
[junit4:junit4] 2> 16218 T2260 oasc.ZkController.register We are http://127.0.0.1:38174/collection1/ and leader is http://127.0.0.1:41035/collection1/
[junit4:junit4] 2> 16218 T2260 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:38174
[junit4:junit4] 2> 16219 T2260 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 16219 T2260 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C365 name=collection1 org.apache.solr.core.SolrCore@e317da url=http://127.0.0.1:38174/collection1 node=127.0.0.1:38174_ C365_STATE=coll:collection1 core:collection1 props:{collection=collection1, node_name=127.0.0.1:38174_, state=down, core=collection1, base_url=http://127.0.0.1:38174}
[junit4:junit4] 2> 16219 T2262 C365 P38174 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 16220 T2262 C365 P38174 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 16220 T2260 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 16220 T2262 C365 P38174 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 16221 T2262 C365 P38174 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 16222 T2262 C365 P38174 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 16222 T2185 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 16222 T2185 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 16223 T2185 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 16284 T2185 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 16287 T2185 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:34006
[junit4:junit4] 2> 16287 T2185 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 16288 T2185 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 16288 T2185 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1364124766665
[junit4:junit4] 2> 16288 T2185 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1364124766665/solr.xml
[junit4:junit4] 2> 16289 T2185 oasc.CoreContainer.<init> New CoreContainer 20216423
[junit4:junit4] 2> 16289 T2185 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1364124766665/'
[junit4:junit4] 2> 16290 T2185 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1364124766665/'
[junit4:junit4] 2> 16309 T2185 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 16309 T2185 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 16310 T2185 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 16310 T2185 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 16311 T2185 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 16311 T2185 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 16311 T2185 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 16312 T2185 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 16312 T2185 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 16312 T2185 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 16318 T2185 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 16328 T2185 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:54134/solr
[junit4:junit4] 2> 16329 T2185 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 16329 T2185 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54134 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@1ab9891
[junit4:junit4] 2> 16331 T2273 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54134. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 16330 T2185 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 16331 T2273 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54134, initiating session
[junit4:junit4] 2> 16331 T2187 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:41191
[junit4:junit4] 2> 16332 T2187 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:41191
[junit4:junit4] 2> 16332 T2189 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9c2bd26f000b with negotiated timeout 20000 for client /127.0.0.1:41191
[junit4:junit4] 2> 16332 T2273 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54134, sessionid = 0x13d9c2bd26f000b, negotiated timeout = 20000
[junit4:junit4] 2> 16333 T2274 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1ab9891 name:ZooKeeperConnection Watcher:127.0.0.1:54134 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 16333 T2185 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 16334 T2190 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d9c2bd26f000b
[junit4:junit4] 2> 16334 T2274 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 16334 T2187 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:41191 which had sessionid 0x13d9c2bd26f000b
[junit4:junit4] 2> 16334 T2185 oaz.ZooKeeper.close Session: 0x13d9c2bd26f000b closed
[junit4:junit4] 2> 16335 T2185 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 16337 T2185 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54134/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@1098467
[junit4:junit4] 2> 16338 T2275 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54134. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 16338 T2185 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 16339 T2275 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54134, initiating session
[junit4:junit4] 2> 16339 T2187 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:41192
[junit4:junit4] 2> 16339 T2187 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:41192
[junit4:junit4] 2> 16340 T2189 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9c2bd26f000c with negotiated timeout 20000 for client /127.0.0.1:41192
[junit4:junit4] 2> 16340 T2275 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54134, sessionid = 0x13d9c2bd26f000c, negotiated timeout = 20000
[junit4:junit4] 2> 16340 T2276 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1098467 name:ZooKeeperConnection Watcher:127.0.0.1:54134/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 16341 T2185 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 16341 T2190 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c2bd26f000c type:create cxid:0x1 zxid:0xa1 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 16343 T2190 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c2bd26f000c type:create cxid:0x3 zxid:0xa2 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 16345 T2185 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 17117 T2208 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 17118 T2208 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "node_name":"127.0.0.1:38174_",
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:38174",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "roles":null}
[junit4:junit4] 2> 17129 T2214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17129 T2243 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17129 T2276 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17129 T2207 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17129 T2227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17129 T2259 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17226 T2220 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {onlyIfLeader=true&state=recovering&coreNodeName=4&action=PREPRECOVERY&version=2&core=collection1&wt=javabin&checkLive=true&nodeName=127.0.0.1:38174_} status=0 QTime=1000
[junit4:junit4] 2> 17347 T2185 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34006_
[junit4:junit4] 2> 17348 T2190 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c2bd26f000c type:delete cxid:0xf zxid:0xa7 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:34006_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:34006_
[junit4:junit4] 2> 17349 T2185 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:34006_
[junit4:junit4] 2> 17350 T2259 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17351 T2214 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 17351 T2207 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 17351 T2276 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 17351 T2214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 17351 T2227 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 17351 T2243 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 17352 T2243 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 17352 T2276 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 17351 T2207 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 17352 T2227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 17352 T2259 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 17357 T2277 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1364124766665/collection1
[junit4:junit4] 2> 17357 T2277 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 17358 T2277 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 17358 T2277 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 17359 T2277 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1364124766665/collection1/'
[junit4:junit4] 2> 17359 T2277 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1364124766665/collection1/lib/README' to classloader
[junit4:junit4] 2> 17360 T2277 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1364124766665/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 17384 T2277 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4] 2> 17420 T2277 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 17422 T2277 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 17427 T2277 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 17758 T2277 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 17764 T2277 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 17766 T2277 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 17780 T2277 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 17784 T2277 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 17788 T2277 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 17789 T2277 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 17790 T2277 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 17791 T2277 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 17792 T2277 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 17792 T2277 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 17792 T2277 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 17793 T2277 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1364124766665/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1364124750443/jetty4/
[junit4:junit4] 2> 17793 T2277 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@11b579d
[junit4:junit4] 2> 17794 T2277 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 17794 T2277 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1364124750443/jetty4
[junit4:junit4] 2> 17795 T2277 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1364124750443/jetty4/index/
[junit4:junit4] 2> 17795 T2277 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1364124750443/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 17795 T2277 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1364124750443/jetty4/index
[junit4:junit4] 2> 17797 T2277 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@113eb22 lockFactory=org.apache.lucene.store.NativeFSLockFactory@357112),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 17798 T2277 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 17800 T2277 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 17800 T2277 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 17801 T2277 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 17802 T2277 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 17802 T2277 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 17803 T2277 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 17803 T2277 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 17804 T2277 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 17804 T2277 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 17808 T2277 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 17812 T2277 oass.SolrIndexSearcher.<init> Opening Searcher@a06d9b main
[junit4:junit4] 2> 17812 T2277 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1364124750443/jetty4/tlog
[junit4:junit4] 2> 17813 T2277 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 17813 T2277 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 17818 T2278 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@a06d9b main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 17820 T2277 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 17820 T2277 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 18634 T2208 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 18635 T2208 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "node_name":"127.0.0.1:34006_",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34006",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "roles":null}
[junit4:junit4] 2> 18635 T2208 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 18635 T2208 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4] 2> 18640 T2227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 18640 T2259 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 18640 T2207 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 18640 T2243 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 18640 T2276 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 18640 T2214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 18822 T2277 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 18822 T2277 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:34006 collection:collection1 shard:shard2
[junit4:junit4] 2> 18826 T2277 oasc.ZkController.register We are http://127.0.0.1:34006/collection1/ and leader is http://127.0.0.1:52380/collection1/
[junit4:junit4] 2> 18826 T2277 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:34006
[junit4:junit4] 2> 18826 T2277 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 18826 T2277 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C366 name=collection1 org.apache.solr.core.SolrCore@f48866 url=http://127.0.0.1:34006/collection1 node=127.0.0.1:34006_ C366_STATE=coll:collection1 core:collection1 props:{collection=collection1, node_name=127.0.0.1:34006_, state=down, core=collection1, base_url=http://127.0.0.1:34006}
[junit4:junit4] 2> 18826 T2279 C366 P34006 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 18828 T2279 C366 P34006 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 18828 T2277 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 18828 T2279 C366 P34006 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 18828 T2279 C366 P34006 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 18829 T2279 C366 P34006 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 18829 T2185 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 18830 T2185 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 18830 T2185 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 18834 T2185 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 18836 T2185 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 18840 T2220 oasc.CoreContainer.create Creating SolrCore 'onenodecollectioncore' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1364124753975/onenodecollectioncore
[junit4:junit4] 2> 18841 T2220 oasc.ZkController.createCollectionZkNode Check for collection zkNode:onenodecollection
[junit4:junit4] 2> 18842 T2220 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:onenodecollection
[junit4:junit4] 2> 18842 T2220 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 18843 T2220 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 18844 T2220 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection
[junit4:junit4] 2> 18846 T2220 oasc.ZkController.readConfigName Load collection config from:/collections/onenodecollection
[junit4:junit4] 2> 18848 T2220 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1364124753975/onenodecollectioncore/'
[junit4:junit4] 2> 18900 T2220 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4] 2> 18930 T2220 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 18931 T2220 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 18934 T2220 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> C365_STATE=coll:collection1 core:collection1 props:{collection=collection1, node_name=127.0.0.1:38174_, state=recovering, shard=shard1, core=collection1, base_url=http://127.0.0.1:38174}
[junit4:junit4] 2> 19227 T2262 C365 P38174 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:41035/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 19227 T2262 C365 P38174 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:38174 START replicas=[http://127.0.0.1:41035/collection1/] nUpdates=100
[junit4:junit4] 2> 19228 T2262 C365 P38174 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 19228 T2262 C365 P38174 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 19228 T2262 C365 P38174 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 19228 T2262 C365 P38174 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 19229 T2262 C365 P38174 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 19229 T2262 C365 P38174 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:41035/collection1/. core=collection1
[junit4:junit4] 2> 19229 T2262 C365 P38174 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C367 name=collection1 org.apache.solr.core.SolrCore@7e023d url=http://127.0.0.1:41035/collection1 node=127.0.0.1:41035_ C367_STATE=coll:collection1 core:collection1 props:{collection=collection1, node_name=127.0.0.1:41035_, state=active, shard=shard1, core=collection1, base_url=http://127.0.0.1:41035, leader=true}
[junit4:junit4] 2> 19233 T2223 C367 P41035 REQ /get {version=2&wt=javabin&qt=/get&distrib=false&getVersions=100} status=0 QTime=0
[junit4:junit4] 2> 19235 T2222 C367 P41035 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 19236 T2222 C367 P41035 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1fb3191 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1ba4243),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 19236 T2222 C367 P41035 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 19237 T2222 C367 P41035 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1fb3191 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1ba4243),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1fb3191 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1ba4243),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> 19237 T2222 C367 P41035 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4] 2> 19238 T2222 C367 P41035 oass.SolrIndexSearcher.<init> Opening Searcher@ca1923 realtime
[junit4:junit4] 2> 19238 T2222 C367 P41035 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 19238 T2222 C367 P41035 /update {waitSearcher=true&version=2&softCommit=false&wt=javabin&commit_end_point=true&commit=true&openSearcher=false} {commit=} 0 3
[junit4:junit4] 2> 19239 T2262 C365 P38174 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 19240 T2262 C365 P38174 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 19243 T2223 C367 P41035 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 19244 T2223 C367 P41035 REQ /replication {command=indexversion&version=2&wt=javabin&qt=/replication} status=0 QTime=4
[junit4:junit4] 2> 19244 T2262 C365 P38174 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 19244 T2262 C365 P38174 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 19245 T2262 C365 P38174 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 19246 T2222 C367 P41035 REQ /replication {generation=2&command=filelist&version=2&wt=javabin&qt=/replication} status=0 QTime=0
[junit4:junit4] 2> 19247 T2262 C365 P38174 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 19248 T2262 C365 P38174 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1364124750443/jetty3/index.20130324073249687
[junit4:junit4] 2> 19248 T2262 C365 P38174 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@14bc680 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1e84bdb) fullCopy=false
[junit4:junit4] 2> 19251 T2221 C367 P41035 REQ /replication {file=segments_2&generation=2&command=filecontent&checksum=true&wt=filestream&qt=/replication} status=0 QTime=1
[junit4:junit4] 2> 19253 T2262 C365 P38174 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 19253 T2262 C365 P38174 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 19253 T2262 C365 P38174 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 19254 T2262 C365 P38174 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@182ecfa lockFactory=org.apache.lucene.store.NativeFSLockFactory@7795a9),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@182ecfa lockFactory=org.apache.lucene.store.NativeFSLockFactory@7795a9),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> 19254 T2262 C365 P38174 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4] 2> 19255 T2262 C365 P38174 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 19255 T2262 C365 P38174 oass.SolrIndexSearcher.<init> Opening Searcher@f4770e main
[junit4:junit4] 2> 19256 T2261 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@f4770e main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 19256 T2262 C365 P38174 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 19256 T2262 C365 P38174 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 19256 T2262 C365 P38174 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 19256 T2262 C365 P38174 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 19258 T2262 C365 P38174 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 19273 T2220 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 19279 T2220 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 19281 T2220 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 19294 T2220 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 19297 T2220 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 19300 T2220 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 19302 T2220 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 19302 T2220 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 19302 T2220 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 19303 T2220 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 19304 T2220 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 19304 T2220 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 19304 T2220 oasc.SolrCore.<init> [onenodecollectioncore] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1364124753975/onenodecollectioncore/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1364124750444/onenodecollection/
[junit4:junit4] 2> 19305 T2220 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@11b579d
[junit4:junit4] 2> 19305 T2220 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 19306 T2220 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1364124750444/onenodecollection
[junit4:junit4] 2> 19306 T2220 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1364124750444/onenodecollection/index/
[junit4:junit4] 2> 19306 T2220 oasc.SolrCore.initIndex WARNING [onenodecollectioncore] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1364124750444/onenodecollection/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 19307 T2220 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1364124750444/onenodecollection/index
[junit4:junit4] 2> 19308 T2220 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@8d5ef6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1076677),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 19308 T2220 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 19310 T2220 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 19310 T2220 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 19311 T2220 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 19311 T2220 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 19312 T2220 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 19312 T2220 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 19312 T2220 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 19313 T2220 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 19313 T2220 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 19315 T2220 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 19317 T2220 oass.SolrIndexSearcher.<init> Opening Searcher@1bb2a74 main
[junit4:junit4] 2> 19318 T2220 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1364124750444/onenodecollection/tlog
[junit4:junit4] 2> 19318 T2220 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 19319 T2220 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 19322 T2283 oasc.SolrCore.registerSearcher [onenodecollectioncore] Registered new searcher Searcher@1bb2a74 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 19323 T2220 oasc.ZkController.publish publishing core=onenodecollectioncore state=down
[junit4:junit4] 2> 20146 T2208 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 20147 T2208 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "node_name":"127.0.0.1:34006_",
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34006",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "roles":null}
[junit4:junit4] 2> 20150 T2208 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "core_node_name":"4",
[junit4:junit4] 2> "node_name":"127.0.0.1:38174_",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:38174",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "roles":null}
[junit4:junit4] 2> 20153 T2208 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:41035_",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "core":"onenodecollectioncore",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:41035",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "roles":"none"}
[junit4:junit4] 2> 20153 T2208 oasc.Overseer$ClusterStateUpdater.createCollection Create collection onenodecollection with numShards 1
[junit4:junit4] 2> 20154 T2208 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 20157 T2243 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 20157 T2276 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 20157 T2207 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 20157 T2227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 20157 T2259 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 20157 T2214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 20324 T2220 oasc.CoreContainer.registerCore registering core: onenodecollectioncore
[junit4:junit4] 2> 20325 T2220 oasc.ZkController.register Register replica - core:onenodecollectioncore address:http://127.0.0.1:41035 collection:onenodecollection shard:shard1
[junit4:junit4] 2> 20326 T2220 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection/leader_elect/shard1/election
[junit4:junit4] 2> 20332 T2190 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c2bd26f0006 type:delete cxid:0xac zxid:0xc2 txntype:-1 reqpath:n/a Error Path:/solr/collections/onenodecollection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/onenodecollection/leaders
[junit4:junit4] 2> 20332 T2220 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 20333 T2190 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c2bd26f0006 type:create cxid:0xad zxid:0xc3 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 20335 T2220 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 20335 T2220 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 20335 T2220 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:41035/onenodecollectioncore/
[junit4:junit4] 2> 20336 T2220 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 20336 T2220 oasc.SyncStrategy.syncToMe http://127.0.0.1:41035/onenodecollectioncore/ has no replicas
[junit4:junit4] 2> 20337 T2220 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:41035/onenodecollectioncore/
[junit4:junit4] 2> 20337 T2220 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection/leaders/shard1
[junit4:junit4] 2> 20340 T2190 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c2bd26f0006 type:create cxid:0xb8 zxid:0xc7 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 20833 T2239 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {onlyIfLeader=true&state=recovering&coreNodeName=5&action=PREPRECOVERY&version=2&core=collection1&wt=javabin&checkLive=true&nodeName=127.0.0.1:34006_} status=0 QTime=2000
[junit4:junit4] 2> 21664 T2208 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 21672 T2214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 21672 T2207 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 21672 T2243 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 21672 T2276 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 21672 T2227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 21672 T2259 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 21695 T2220 oasc.ZkController.register We are http://127.0.0.1:41035/onenodecollectioncore/ and leader is http://127.0.0.1:41035/onenodecollectioncore/
[junit4:junit4] 2> 21695 T2220 oasc.ZkController.register No LogReplay needed for core=onenodecollectioncore baseURL=http://127.0.0.1:41035
[junit4:junit4] 2> 21695 T2220 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 21696 T2220 oasc.ZkController.publish publishing core=onenodecollectioncore state=active
[junit4:junit4] 2> 21697 T2220 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 21698 T2220 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {collection=onenodecollection&action=CREATE&version=2&numShards=1&name=onenodecollectioncore&dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1364124750444/onenodecollection&wt=javabin&roles=none} status=0 QTime=2858
[junit4:junit4] 2> 21699 T2185 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: onenodecollection failOnTimeout:true timeout (sec):230
[junit4:junit4] 2> 21700 T2185 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 22701 T2185 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C366_STATE=coll:collection1 core:collection1 props:{collection=collection1, node_name=127.0.0.1:34006_, state=recovering, shard=shard2, core=collection1, base_url=http://127.0.0.1:34006}
[junit4:junit4] 2> 22834 T2279 C366 P34006 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:52380/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 22834 T2279 C366 P34006 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:34006 START replicas=[http://127.0.0.1:52380/collection1/] nUpdates=100
[junit4:junit4] 2> 22835 T2279 C366 P34006 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 22835 T2279 C366 P34006 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 22835 T2279 C366 P34006 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 22835 T2279 C366 P34006 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 22835 T2279 C366 P34006 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 22836 T2279 C366 P34006 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:52380/collection1/. core=collection1
[junit4:junit4] 2> 22836 T2279 C366 P34006 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C368 name=collection1 org.apache.solr.core.SolrCore@392959 url=http://127.0.0.1:52380/collection1 node=127.0.0.1:52380_ C368_STATE=coll:collection1 core:collection1 props:{collection=collection1, node_name=127.0.0.1:52380_, state=active, shard=shard2, core=collection1, base_url=http://127.0.0.1:52380, leader=true}
[junit4:junit4] 2> 22840 T2236 C368 P52380 REQ /get {version=2&wt=javabin&qt=/get&distrib=false&getVersions=100} status=0 QTime=1
[junit4:junit4] 2> 22840 T2238 C368 P52380 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 22841 T2238 C368 P52380 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@a65a4a lockFactory=org.apache.lucene.store.NativeFSLockFactory@17d94e0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 22842 T2238 C368 P52380 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 22842 T2238 C368 P52380 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@a65a4a lockFactory=org.apache.lucene.store.NativeFSLockFactory@17d94e0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@a65a4a lockFactory=org.apache.lucene.store.NativeFSLockFactory@17d94e0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> 22842 T2238 C368 P52380 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4] 2> 22843 T2238 C368 P52380 oass.SolrIndexSearcher.<init> Opening Searcher@1370f0a realtime
[junit4:junit4] 2> 22843 T2238 C368 P52380 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 22843 T2238 C368 P52380 /update {waitSearcher=true&version=2&softCommit=false&wt=javabin&commit_end_point=true&commit=true&openSearcher=false} {commit=} 0 3
[junit4:junit4] 2> 22844 T2279 C366 P34006 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 22844 T2279 C366 P34006 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 22846 T2237 C368 P52380 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 22846 T2237 C368 P52380 REQ /replication {command=indexversion&version=2&wt=javabin&qt=/replication} status=0 QTime=1
[junit4:junit4] 2> 22847 T2279 C366 P34006 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 22847 T2279 C366 P34006 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 22847 T2279 C366 P34006 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 22848 T2236 C368 P52380 REQ /replication {generation=2&command=filelist&version=2&wt=javabin&qt=/replication} status=0 QTime=0
[junit4:junit4] 2> 22849 T2279 C366 P34006 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 22850 T2279 C366 P34006 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1364124750443/jetty4/index.20130324073253289
[junit4:junit4] 2> 22850 T2279 C366 P34006 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@b67628 lockFactory=org.apache.lucene.store.NativeFSLockFactory@f1a722) fullCopy=false
[junit4:junit4] 2> 22852 T2238 C368 P52380 REQ /replication {file=segments_2&generation=2&command=filecontent&checksum=true&wt=filestream&qt=/replication} status=0 QTime=0
[junit4:junit4] 2> 22854 T2279 C366 P34006 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 22854 T2279 C366 P34006 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 22854 T2279 C366 P34006 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 22855 T2279 C366 P34006 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@113eb22 lockFactory=org.apache.lucene.store.NativeFSLockFactory@357112),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@113eb22 lockFactory=org.apache.lucene.store.NativeFSLockFactory@357112),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> 22856 T2279 C366 P34006 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4] 2> 22856 T2279 C366 P34006 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 22856 T2279 C366 P34006 oass.SolrIndexSearcher.<init> Opening Searcher@9ea1a1 main
[junit4:junit4] 2> 22857 T2278 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@9ea1a1 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 22857 T2279 C366 P34006 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 22857 T2279 C366 P34006 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 22858 T2279 C366 P34006 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 22858 T2279 C366 P34006 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 22859 T2279 C366 P34006 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 23180 T2208 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 23181 T2208 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:41035__onenodecollectioncore",
[junit4:junit4] 2> "node_name":"127.0.0.1:41035_",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"onenodecollectioncore",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:41035",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "roles":"none"}
[junit4:junit4] 2> 23184 T2208 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "core_node_name":"5",
[junit4:junit4] 2> "node_name":"127.0.0.1:34006_",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34006",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "roles":null}
[junit4:junit4] 2> 23187 T2243 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 23187 T2259 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 23187 T2227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 23187 T2207 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 23187 T2214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 23187 T2276 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 23703 T2185 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 23704 T2185 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: onenodecollection
[junit4:junit4] 2> 23705 T2185 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 23713 T2185 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54134 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@10bbb81
[junit4:junit4] 2> 23714 T2185 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 23715 T2286 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:54134. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 23715 T2286 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:54134, initiating session
[junit4:junit4] 2> 23715 T2187 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:41203
[junit4:junit4] 2> 23715 T2187 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:41203
[junit4:junit4] 2> 23716 T2189 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9c2bd26f000d with negotiated timeout 10000 for client /127.0.0.1:41203
[junit4:junit4] 2> 23716 T2286 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:54134, sessionid = 0x13d9c2bd26f000d, negotiated timeout = 10000
[junit4:junit4] 2> 23717 T2287 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@10bbb81 name:ZooKeeperConnection Watcher:127.0.0.1:54134 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 23717 T2185 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 23763 T2190 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d9c2bd26f000d
[junit4:junit4] 2> 23764 T2287 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 23764 T2187 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:41203 which had sessionid 0x13d9c2bd26f000d
[junit4:junit4] 2> 23764 T2185 oaz.ZooKeeper.close Session: 0x13d9c2bd26f000d closed
[junit4:junit4] 2> 23765 T2190 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d9c2bd26f0004
[junit4:junit4] 2> 23765 T2214 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 23765 T2187 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:41176 which had sessionid 0x13d9c2bd26f0004
[junit4:junit4] 2> 23765 T2185 oaz.ZooKeeper.close Session: 0x13d9c2bd26f0004 closed
[junit4:junit4] 2> 23767 T2185 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 50017
[junit4:junit4] 2> 23768 T2185 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=20301957
[junit4:junit4] 2> 24692 T2208 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 24693 T2208 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "node_name":"127.0.0.1:50017_",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50017",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "roles":null}
[junit4:junit4] 2> 24698 T2227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 24698 T2276 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 24698 T2259 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 24698 T2243 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 24698 T2207 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 25769 T2185 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 25770 T2185 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 25772 T2185 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@15d681
[junit4:junit4] 2> 25775 T2185 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 25775 T2185 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 25776 T2185 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 25776 T2185 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 25777 T2185 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 25778 T2185 oasc.CachingDirectoryFactory.close Closing directory when closing factory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1364124750443/control/data
[junit4:junit4] 2> 25778 T2185 oasc.CachingDirectoryFactory.closeDirectory Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1364124750443/control/data
[junit4:junit4] 2> 25778 T2185 oasc.CachingDirectoryFactory.close Closing directory when closing factory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1364124750443/control/data/index
[junit4:junit4] 2> 25778 T2185 oasc.CachingDirectoryFactory.closeDirectory Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1364124750443/control/data/index
[junit4:junit4] 2> 25779 T2208 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89399279645294595-127.0.0.1:50017_-n_0000000000) am no longer a leader.
[junit4:junit4] 2> 25780 T2190 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d9c2bd26f0003
[junit4:junit4] 2> 25781 T2207 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 25781 T2227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 25781 T2276 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 25781 T2187 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:41175 which had sessionid 0x13d9c2bd26f0003
[junit4:junit4] 2> 25781 T2243 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 25781 T2259 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 25781 T2243 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 25781 T2185 oaz.ZooKeeper.close Session: 0x13d9c2bd26f0003 closed
[junit4:junit4] 2> 25782 T2259 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 25783 T2276 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 25783 T2227 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 25785 T2190 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c2bd26f0006 type:delete cxid:0xc9 zxid:0xe3 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4] 2> 25785 T2227 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 25787 T2227 oasc.Overseer.start Overseer (id=89399279645294598-127.0.0.1:41035_-n_0000000001) starting
[junit4:junit4] 2> 25787 T2190 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c2bd26f0006 type:create cxid:0xce zxid:0xe5 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 25788 T2190 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c2bd26f0006 type:create cxid:0xcf zxid:0xe6 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 25789 T2190 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c2bd26f0006 type:create cxid:0xd1 zxid:0xe7 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 25790 T2190 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c2bd26f0006 type:create cxid:0xd3 zxid:0xe8 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 25791 T2289 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 25792 T2288 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 25793 T2288 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 25794 T2288 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "core_node_name":"1",
[junit4:junit4] 2> "node_name":"127.0.0.1:50017_",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50017",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "roles":null}
[junit4:junit4] 2> 25797 T2276 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 25797 T2259 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 25797 T2243 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 25797 T2227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 25804 T2185 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
[junit4:junit4] 2> 25857 T2185 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 41035
[junit4:junit4] 2> 25857 T2185 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=31263381
[junit4:junit4] 2> 27081 T2207 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 27081 T2207 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 27081 T2207 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 27301 T2288 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 27302 T2288 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "core_node_name":"2",
[junit4:junit4] 2> "node_name":"127.0.0.1:41035_",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:41035",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "roles":null}
[junit4:junit4] 2> 27307 T2288 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:41035__onenodecollectioncore",
[junit4:junit4] 2> "node_name":"127.0.0.1:41035_",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "core":"onenodecollectioncore",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:41035",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "roles":"none"}
[junit4:junit4] 2> 27313 T2243 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 27313 T2259 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 27313 T2227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 27313 T2276 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 28860 T2185 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 28861 T2185 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 28862 T2185 oasc.ZkController.publish publishing core=onenodecollectioncore state=down
[junit4:junit4] 2> 28864 T2185 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@7e023d
[junit4:junit4] 2> 28869 T2185 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 28869 T2185 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 28870 T2185 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 28870 T2185 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 28871 T2185 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 28872 T2185 oasc.CachingDirectoryFactory.close Closing directory when closing factory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1364124750443/jetty1
[junit4:junit4] 2> 28872 T2185 oasc.CachingDirectoryFactory.closeDirectory Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1364124750443/jetty1
[junit4:junit4] 2> 28872 T2185 oasc.CachingDirectoryFactory.close Closing directory when closing factory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1364124750443/jetty1/index
[junit4:junit4] 2> 28873 T2185 oasc.CachingDirectoryFactory.closeDirectory Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1364124750443/jetty1/index
[junit4:junit4] 2> 28873 T2185 oasc.SolrCore.close [onenodecollectioncore] CLOSING SolrCore org.apache.solr.core.SolrCore@166127c
[junit4:junit4] 2> 28877 T2185 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 28877 T2185 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 28877 T2185 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 28878 T2185 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 28878 T2185 oasc.SolrCore.closeSearcher [onenodecollectioncore] Closing main searcher on request.
[junit4:junit4] 2> 28879 T2185 oasc.CachingDirectoryFactory.close Closing directory when closing factory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1364124750444/onenodecollection
[junit4:junit4] 2> 28879 T2185 oasc.CachingDirectoryFactory.closeDirectory Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1364124750444/onenodecollection
[junit4:junit4] 2> 28879 T2185 oasc.CachingDirectoryFactory.close Closing directory when closing factory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1364124750444/onenodecollection/index
[junit4:junit4] 2> 28880 T2185 oasc.CachingDirectoryFactory.closeDirectory Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1364124750444/onenodecollection/index
[junit4:junit4] 2> 28880 T2288 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89399279645294598-127.0.0.1:41035_-n_0000000001) am no longer a leader.
[junit4:junit4] 2> 28881 T2190 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d9c2bd26f0006
[junit4:junit4] 2> 28882 T2276 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 28882 T2243 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 28882 T2227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 28882 T2187 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:41178 which had sessionid 0x13d9c2bd26f0006
[junit4:junit4] 2> 28882 T2227 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 28883 T2227 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 28883 T2227 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 28883 T2190 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c2bd26f000a type:delete cxid:0x5e zxid:0xf9 txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard1 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard1
[junit4:junit4] 2> 28882 T2185 oaz.ZooKeeper.close Session: 0x13d9c2bd26f0006 closed
[junit4:junit4] 2> 28883 T2259 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 28884 T2190 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c2bd26f000a type:create cxid:0x5f zxid:0xfa txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 28885 T2276 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 28885 T2243 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 28886 T2259 oasc.ShardLeaderElectionContext.shouldIBeLeader Checking if I should try and be the leader.
[junit4:junit4] 2> 28886 T2259 oasc.ShardLeaderElectionContext.shouldIBeLeader My last published State was Active, it's okay to be the leader.
[junit4:junit4] 2> 28886 T2259 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 28886 T2190 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c2bd26f0008 type:delete cxid:0x7f zxid:0xfc txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4] 2> 28886 T2259 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:38174/collection1/
[junit4:junit4] 2> 28886 T2259 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:38174 START replicas=[http://127.0.0.1:41035/collection1/] nUpdates=100
[junit4:junit4] 2> 28886 T2243 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 28886 T2259 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:38174 DONE. We have no versions. sync failed.
[junit4:junit4] 2> 28887 T2259 oasc.SyncStrategy.syncReplicas Leader's attempt to sync with shard failed, moving to the next canidate
[junit4:junit4] 2> 28887 T2259 oasc.ShardLeaderElectionContext.runLeaderProcess We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4:junit4] 2> 28887 T2259 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:38174/collection1/
[junit4:junit4] 2> 28887 T2259 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 28888 T2243 oasc.Overseer.start Overseer (id=89399279645294600-127.0.0.1:52380_-n_0000000002) starting
[junit4:junit4] 2> 28888 T2190 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c2bd26f0008 type:create cxid:0x84 zxid:0xfe txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 28889 T2190 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9c2bd26f0008 type:create cxid:0x85 zxid:0xff txntype:
[...truncated too long message...]
DistribSearch <<<
[junit4:junit4] > Throwable #1: org.apache.solr.common.SolrException: Server at http://127.0.0.1:52380/onenodecollectioncore returned non ok status:404, message:Can not find: /onenodecollectioncore/update
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([995C80BF3A3661E9:18BA0EA74D6901D5]:0)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:375)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181)
[junit4:junit4] > at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
[junit4:junit4] > at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:116)
[junit4:junit4] > at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:102)
[junit4:junit4] > at org.apache.solr.cloud.BasicDistributedZk2Test.testNodeWithoutCollectionForwarding(BasicDistributedZk2Test.java:197)
[junit4:junit4] > at org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:89)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
[junit4:junit4] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4:junit4] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[junit4:junit4] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4:junit4] > at java.lang.reflect.Method.invoke(Method.java:487)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4] > at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4:junit4] > at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4:junit4] > at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4] > at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
[junit4:junit4] > at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4] > at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4] > at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4] > at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4] > at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4] > at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4] > at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2> 32242 T2185 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 32246 T2184 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 3 leaked thread(s).
[junit4:junit4] 2> 32358 T2259 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 32358 T2259 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 32358 T2259 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 32359 T2259 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 32560 T2188 oazs.SessionTrackerImpl.run SessionTrackerImpl exited loop!
[junit4:junit4] 2> NOTE: test params are: codec=SimpleText, sim=DefaultSimilarity, locale=tr_TR, timezone=America/Indiana/Indianapolis
[junit4:junit4] 2> NOTE: Linux 3.2.0-39-generic i386/Oracle Corporation 1.8.0-ea (32-bit)/cpus=8,threads=3,free=56204528,total=316669952
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TermVectorComponentDistributedTest, TestQueryTypes, TestCSVLoader, CurrencyFieldXmlFileTest, CollectionsAPIDistributedZkTest, SpellCheckCollatorTest, XsltUpdateRequestHandlerTest, TestFieldTypeCollectionResource, HighlighterTest, SignatureUpdateProcessorFactoryTest, TestLazyCores, SortByFunctionTest, TestSolr4Spatial, OpenCloseCoreStressTest, TestBinaryField, CachingDirectoryFactoryTest, ChaosMonkeySafeLeaderTest, SolrCmdDistributorTest, CoreContainerCoreInitFailuresTest, NoCacheHeaderTest, IndexSchemaRuntimeFieldTest, ConvertedLegacyTest, TimeZoneUtilsTest, TestJmxIntegration, UniqFieldsUpdateProcessorFactoryTest, OpenExchangeRatesOrgProviderTest, SpellCheckComponentTest, StatsComponentTest, DistanceFunctionTest, TestSolrXMLSerializer, TestFastOutputStream, TestZkChroot, TestLuceneMatchVersion, TestFoldingMultitermQuery, TestCollationField, TestNumberUtils, TestSuggestSpellingConverter, TestArbitraryIndexDir, TestMultiCoreConfBootstrap, SuggesterTSTTest, MultiTermTest, DOMUtilTest, TestAnalyzedSuggestions, FullSolrCloudDistribCmdsTest, RequestHandlersTest, TestStressReorder, PathHierarchyTokenizerFactoryTest, TestSolrJ, HighlighterConfigTest, TestRTGBase, ZkSolrClientTest, JSONWriterTest, TestPartialUpdateDeduplication, ZkNodePropsTest, TestUpdate, TestMergePolicyConfig, SimplePostToolTest, SolrCoreCheckLockOnStartupTest, XmlUpdateRequestHandlerTest, TestHashPartitioner, TestAtomicUpdateErrorCases, NotRequiredUniqueKeyTest, TestCSVResponseWriter, IndexBasedSpellCheckerTest, BasicDistributedZk2Test]
[junit4:junit4] Completed on J1 in 32.78s, 1 test, 1 error <<< FAILURES!
[...truncated 445 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:381: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:361: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:183: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:449: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1212: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:876: There were test failures: 273 suites, 1149 tests, 1 error, 13 ignored (7 assumptions)
Total time: 40 minutes 33 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Description set: Java: 32bit/jdk1.8.0-ea-b82 -server -XX:+UseG1GC -XX:MarkStackSize=256K
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b82) - Build #
4804 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/4804/
Java: 32bit/jdk1.8.0-ea-b82 -server -XX:+UseG1GC -XX:MarkStackSize=256K
2 tests failed.
FAILED: org.apache.solr.cloud.BasicDistributedZk2Test.testDistribSearch
Error Message:
Server at http://127.0.0.1:52405/onenodecollectioncore returned non ok status:404, message:Can not find: /onenodecollectioncore/update
Stack Trace:
org.apache.solr.common.SolrException: Server at http://127.0.0.1:52405/onenodecollectioncore returned non ok status:404, message:Can not find: /onenodecollectioncore/update
at __randomizedtesting.SeedInfo.seed([9EA55DA4301C2E2D:1F43D3BC47434E11]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:375)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181)
at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:116)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:102)
at org.apache.solr.cloud.BasicDistributedZk2Test.testNodeWithoutCollectionForwarding(BasicDistributedZk2Test.java:197)
at org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:89)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:487)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
FAILED: org.apache.solr.cloud.UnloadDistributedZkTest.testDistribSearch
Error Message:
Still found shard2 in collection test_unload_shard_and_collection
Stack Trace:
java.lang.AssertionError: Still found shard2 in collection test_unload_shard_and_collection
at __randomizedtesting.SeedInfo.seed([9EA55DA4301C2E2D:1F43D3BC47434E11]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.UnloadDistributedZkTest.testUnloadShardAndCollection(UnloadDistributedZkTest.java:127)
at org.apache.solr.cloud.UnloadDistributedZkTest.doTest(UnloadDistributedZkTest.java:77)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:487)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Build Log:
[...truncated 9186 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZk2Test
[junit4:junit4] 2> 0 T631 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4] 2> 3 T631 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1364119264196
[junit4:junit4] 2> 4 T631 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 4 T632 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 6 T632 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4] 2> 6 T632 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4] 2> 6 T632 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4] 2> 7 T632 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4] 2> 8 T632 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-1364119264195/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4] 2> 104 T631 oasc.ZkTestServer.run start zk server on port:41151
[junit4:junit4] 2> 105 T631 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:41151 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@a9d32a
[junit4:junit4] 2> 106 T631 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 106 T637 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:41151. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 107 T637 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:41151, initiating session
[junit4:junit4] 2> 107 T633 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:44974
[junit4:junit4] 2> 108 T633 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:44974
[junit4:junit4] 2> 108 T635 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4] 2> 143 T635 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9bd81bc90000 with negotiated timeout 10000 for client /127.0.0.1:44974
[junit4:junit4] 2> 143 T637 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:41151, sessionid = 0x13d9bd81bc90000, negotiated timeout = 10000
[junit4:junit4] 2> 144 T638 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a9d32a name:ZooKeeperConnection Watcher:127.0.0.1:41151 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 145 T631 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 145 T631 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 150 T636 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d9bd81bc90000
[junit4:junit4] 2> 151 T638 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 152 T633 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:44974 which had sessionid 0x13d9bd81bc90000
[junit4:junit4] 2> 151 T631 oaz.ZooKeeper.close Session: 0x13d9bd81bc90000 closed
[junit4:junit4] 2> 152 T631 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:41151/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@6e308f
[junit4:junit4] 2> 154 T639 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:41151. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 154 T631 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 155 T639 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:41151, initiating session
[junit4:junit4] 2> 155 T633 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:44975
[junit4:junit4] 2> 156 T633 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:44975
[junit4:junit4] 2> 157 T635 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9bd81bc90001 with negotiated timeout 10000 for client /127.0.0.1:44975
[junit4:junit4] 2> 157 T639 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:41151, sessionid = 0x13d9bd81bc90001, negotiated timeout = 10000
[junit4:junit4] 2> 158 T640 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6e308f name:ZooKeeperConnection Watcher:127.0.0.1:41151/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 158 T631 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 159 T631 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 165 T631 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 170 T631 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 174 T631 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 178 T631 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 179 T631 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 185 T631 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 186 T631 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 192 T631 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 193 T631 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 198 T631 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 199 T631 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 203 T631 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 204 T631 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 209 T631 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 209 T631 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 214 T631 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 214 T631 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 220 T631 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 221 T631 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 226 T631 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 227 T631 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 232 T636 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d9bd81bc90001
[junit4:junit4] 2> 233 T640 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 233 T633 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:44975 which had sessionid 0x13d9bd81bc90001
[junit4:junit4] 2> 233 T631 oaz.ZooKeeper.close Session: 0x13d9bd81bc90001 closed
[junit4:junit4] 2> 301 T631 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 308 T631 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:60538
[junit4:junit4] 2> 309 T631 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 309 T631 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 309 T631 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1364119264426
[junit4:junit4] 2> 310 T631 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1364119264426/solr.xml
[junit4:junit4] 2> 310 T631 oasc.CoreContainer.<init> New CoreContainer 4602512
[junit4:junit4] 2> 310 T631 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1364119264426/'
[junit4:junit4] 2> 311 T631 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1364119264426/'
[junit4:junit4] 2> 327 T631 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 327 T631 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 328 T631 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 328 T631 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 328 T631 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 329 T631 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 329 T631 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 329 T631 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 329 T631 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 330 T631 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 338 T631 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 347 T631 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:41151/solr
[junit4:junit4] 2> 348 T631 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 348 T631 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:41151 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@18a1efd
[junit4:junit4] 2> 349 T631 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 349 T650 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:41151. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 350 T650 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:41151, initiating session
[junit4:junit4] 2> 350 T633 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:44976
[junit4:junit4] 2> 351 T633 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:44976
[junit4:junit4] 2> 352 T635 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9bd81bc90002 with negotiated timeout 20000 for client /127.0.0.1:44976
[junit4:junit4] 2> 352 T650 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:41151, sessionid = 0x13d9bd81bc90002, negotiated timeout = 20000
[junit4:junit4] 2> 353 T651 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@18a1efd name:ZooKeeperConnection Watcher:127.0.0.1:41151 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 353 T631 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 355 T636 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d9bd81bc90002
[junit4:junit4] 2> 356 T651 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 356 T633 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:44976 which had sessionid 0x13d9bd81bc90002
[junit4:junit4] 2> 356 T631 oaz.ZooKeeper.close Session: 0x13d9bd81bc90002 closed
[junit4:junit4] 2> 357 T631 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 360 T631 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:41151/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@14cfbcd
[junit4:junit4] 2> 361 T652 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:41151. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 361 T631 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 361 T652 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:41151, initiating session
[junit4:junit4] 2> 362 T633 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:44977
[junit4:junit4] 2> 362 T633 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:44977
[junit4:junit4] 2> 363 T635 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9bd81bc90003 with negotiated timeout 20000 for client /127.0.0.1:44977
[junit4:junit4] 2> 364 T652 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:41151, sessionid = 0x13d9bd81bc90003, negotiated timeout = 20000
[junit4:junit4] 2> 364 T653 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14cfbcd name:ZooKeeperConnection Watcher:127.0.0.1:41151/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 365 T631 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 367 T631 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 372 T636 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9bd81bc90003 type:create cxid:0x7 zxid:0x1b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 374 T631 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 378 T631 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 383 T631 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60538_
[junit4:junit4] 2> 384 T636 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9bd81bc90003 type:delete cxid:0x12 zxid:0x1e txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:60538_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:60538_
[junit4:junit4] 2> 386 T631 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:60538_
[junit4:junit4] 2> 393 T631 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 401 T636 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9bd81bc90003 type:delete cxid:0x20 zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4] 2> 402 T631 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 407 T631 oasc.Overseer.start Overseer (id=89398920098676739-127.0.0.1:60538_-n_0000000000) starting
[junit4:junit4] 2> 408 T636 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9bd81bc90003 type:create cxid:0x25 zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 410 T636 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9bd81bc90003 type:create cxid:0x26 zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 412 T636 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9bd81bc90003 type:create cxid:0x28 zxid:0x27 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 414 T631 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 418 T636 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9bd81bc90003 type:create cxid:0x2e zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 421 T655 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 422 T631 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 426 T631 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 430 T631 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 435 T654 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 439 T656 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1364119264426/collection1
[junit4:junit4] 2> 439 T656 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 440 T656 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 441 T656 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 443 T656 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1364119264426/collection1/'
[junit4:junit4] 2> 444 T656 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1364119264426/collection1/lib/README' to classloader
[junit4:junit4] 2> 444 T656 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1364119264426/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 483 T656 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4] 2> 510 T656 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 512 T656 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 516 T656 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 844 T656 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 850 T656 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 852 T656 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 863 T656 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 866 T656 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 869 T656 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 870 T656 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 870 T656 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 870 T656 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 871 T656 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 871 T656 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 871 T656 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 872 T656 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1364119264426/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1364119264195/control/data/
[junit4:junit4] 2> 872 T656 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@555231
[junit4:junit4] 2> 872 T656 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 873 T656 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1364119264195/control/data
[junit4:junit4] 2> 873 T656 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1364119264195/control/data/index/
[junit4:junit4] 2> 873 T656 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1364119264195/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 873 T656 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1364119264195/control/data/index
[junit4:junit4] 2> 875 T656 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@16777a0 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1ad52d2),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 875 T656 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 876 T656 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 877 T656 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 877 T656 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 877 T656 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 878 T656 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 878 T656 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 878 T656 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 878 T656 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 879 T656 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 881 T656 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 883 T656 oass.SolrIndexSearcher.<init> Opening Searcher@10ecee6 main
[junit4:junit4] 2> 883 T656 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1364119264195/control/data/tlog
[junit4:junit4] 2> 884 T656 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 884 T656 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 887 T657 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@10ecee6 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 888 T656 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 889 T656 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1942 T654 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1944 T654 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "node_name":"127.0.0.1:60538_",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "base_url":"http://127.0.0.1:60538"}
[junit4:junit4] 2> 1944 T654 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 1944 T654 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 1952 T653 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 2890 T656 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 2890 T656 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:60538 collection:control_collection shard:shard1
[junit4:junit4] 2> 2892 T656 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 2905 T636 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9bd81bc90003 type:delete cxid:0x82 zxid:0x35 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4] 2> 2906 T656 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 2907 T636 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9bd81bc90003 type:create cxid:0x83 zxid:0x36 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 2912 T656 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 2912 T656 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 2912 T656 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:60538/collection1/
[junit4:junit4] 2> 2913 T656 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 2913 T656 oasc.SyncStrategy.syncToMe http://127.0.0.1:60538/collection1/ has no replicas
[junit4:junit4] 2> 2913 T656 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:60538/collection1/
[junit4:junit4] 2> 2914 T656 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 2922 T636 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9bd81bc90003 type:create cxid:0x8e zxid:0x3a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 3460 T654 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 3475 T653 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 3530 T656 oasc.ZkController.register We are http://127.0.0.1:60538/collection1/ and leader is http://127.0.0.1:60538/collection1/
[junit4:junit4] 2> 3530 T656 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:60538
[junit4:junit4] 2> 3530 T656 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 3531 T656 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 3531 T656 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 3534 T656 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 3536 T631 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 3537 T631 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 3537 T631 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 3545 T631 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 3546 T631 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:41151/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@ddc335
[junit4:junit4] 2> 3548 T659 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:41151. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 3548 T631 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 3548 T659 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:41151, initiating session
[junit4:junit4] 2> 3549 T633 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:44985
[junit4:junit4] 2> 3549 T633 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:44985
[junit4:junit4] 2> 3550 T635 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9bd81bc90004 with negotiated timeout 10000 for client /127.0.0.1:44985
[junit4:junit4] 2> 3551 T659 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:41151, sessionid = 0x13d9bd81bc90004, negotiated timeout = 10000
[junit4:junit4] 2> 3551 T660 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ddc335 name:ZooKeeperConnection Watcher:127.0.0.1:41151/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 3552 T631 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 3555 T631 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 3559 T631 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 3633 T631 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 3636 T631 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:44347
[junit4:junit4] 2> 3637 T631 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 3637 T631 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 3638 T631 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1364119267751
[junit4:junit4] 2> 3638 T631 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1364119267751/solr.xml
[junit4:junit4] 2> 3638 T631 oasc.CoreContainer.<init> New CoreContainer 11485985
[junit4:junit4] 2> 3639 T631 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1364119267751/'
[junit4:junit4] 2> 3639 T631 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1364119267751/'
[junit4:junit4] 2> 3655 T631 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 3655 T631 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 3656 T631 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 3656 T631 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 3656 T631 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 3657 T631 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 3657 T631 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 3657 T631 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 3658 T631 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 3658 T631 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 3664 T631 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 3673 T631 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:41151/solr
[junit4:junit4] 2> 3673 T631 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 3674 T631 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:41151 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@1c70015
[junit4:junit4] 2> 3674 T631 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 3675 T670 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:41151. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 3676 T670 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:41151, initiating session
[junit4:junit4] 2> 3676 T633 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:44986
[junit4:junit4] 2> 3676 T633 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:44986
[junit4:junit4] 2> 3678 T635 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9bd81bc90005 with negotiated timeout 20000 for client /127.0.0.1:44986
[junit4:junit4] 2> 3678 T670 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:41151, sessionid = 0x13d9bd81bc90005, negotiated timeout = 20000
[junit4:junit4] 2> 3678 T671 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c70015 name:ZooKeeperConnection Watcher:127.0.0.1:41151 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 3679 T631 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 3680 T636 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d9bd81bc90005
[junit4:junit4] 2> 3681 T671 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 3681 T633 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:44986 which had sessionid 0x13d9bd81bc90005
[junit4:junit4] 2> 3681 T631 oaz.ZooKeeper.close Session: 0x13d9bd81bc90005 closed
[junit4:junit4] 2> 3682 T631 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 3685 T631 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:41151/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@c0d773
[junit4:junit4] 2> 3686 T672 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:41151. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 3685 T631 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 3686 T672 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:41151, initiating session
[junit4:junit4] 2> 3686 T633 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:44987
[junit4:junit4] 2> 3687 T633 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:44987
[junit4:junit4] 2> 3688 T635 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9bd81bc90006 with negotiated timeout 20000 for client /127.0.0.1:44987
[junit4:junit4] 2> 3688 T672 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:41151, sessionid = 0x13d9bd81bc90006, negotiated timeout = 20000
[junit4:junit4] 2> 3688 T673 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c0d773 name:ZooKeeperConnection Watcher:127.0.0.1:41151/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 3689 T631 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 3690 T636 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9bd81bc90006 type:create cxid:0x1 zxid:0x48 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 3692 T636 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9bd81bc90006 type:create cxid:0x3 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 3695 T631 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 4700 T631 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44347_
[junit4:junit4] 2> 4701 T636 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9bd81bc90006 type:delete cxid:0xd zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:44347_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:44347_
[junit4:junit4] 2> 4703 T631 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:44347_
[junit4:junit4] 2> 4708 T653 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 4708 T660 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 4708 T673 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 4711 T653 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 4718 T674 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1364119267751/collection1
[junit4:junit4] 2> 4718 T674 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 4720 T674 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 4720 T674 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 4722 T674 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1364119267751/collection1/'
[junit4:junit4] 2> 4723 T674 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1364119267751/collection1/lib/README' to classloader
[junit4:junit4] 2> 4723 T674 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1364119267751/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 4752 T674 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4] 2> 4779 T674 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 4780 T674 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 4784 T674 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 4983 T654 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 4984 T654 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "node_name":"127.0.0.1:60538_",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:60538__collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:60538"}
[junit4:junit4] 2> 4989 T660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 4989 T673 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 4989 T653 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 5105 T674 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 5110 T674 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 5112 T674 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 5132 T674 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 5136 T674 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 5142 T674 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 5144 T674 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 5144 T674 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 5145 T674 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 5147 T674 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 5147 T674 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 5148 T674 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 5148 T674 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1364119267751/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1364119264195/jetty1/
[junit4:junit4] 2> 5149 T674 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@555231
[junit4:junit4] 2> 5149 T674 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 5150 T674 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1364119264195/jetty1
[junit4:junit4] 2> 5150 T674 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1364119264195/jetty1/index/
[junit4:junit4] 2> 5151 T674 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1364119264195/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 5151 T674 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1364119264195/jetty1/index
[junit4:junit4] 2> 5154 T674 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@132412 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1bd372f),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 5154 T674 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 5157 T674 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 5158 T674 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 5159 T674 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 5160 T674 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 5161 T674 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 5161 T674 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 5161 T674 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 5162 T674 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 5163 T674 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 5167 T674 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 5172 T674 oass.SolrIndexSearcher.<init> Opening Searcher@163e21c main
[junit4:junit4] 2> 5172 T674 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1364119264195/jetty1/tlog
[junit4:junit4] 2> 5173 T674 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 5173 T674 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 5181 T675 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@163e21c main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 5185 T674 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 5185 T674 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 6495 T654 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 6497 T654 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "node_name":"127.0.0.1:44347_",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "base_url":"http://127.0.0.1:44347"}
[junit4:junit4] 2> 6497 T654 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4] 2> 6497 T654 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4] 2> 6505 T653 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 6505 T660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 6506 T673 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 7187 T674 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 7188 T674 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:44347 collection:collection1 shard:shard2
[junit4:junit4] 2> 7189 T674 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 7202 T636 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9bd81bc90006 type:delete cxid:0x4a zxid:0x5a txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4] 2> 7203 T674 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 7204 T636 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9bd81bc90006 type:create cxid:0x4b zxid:0x5b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 7208 T674 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 7208 T674 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 7208 T674 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:44347/collection1/
[junit4:junit4] 2> 7209 T674 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 7209 T674 oasc.SyncStrategy.syncToMe http://127.0.0.1:44347/collection1/ has no replicas
[junit4:junit4] 2> 7209 T674 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:44347/collection1/
[junit4:junit4] 2> 7210 T674 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 7217 T636 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9bd81bc90006 type:create cxid:0x56 zxid:0x5f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 8013 T654 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 8026 T653 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 8026 T660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 8026 T673 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 8074 T674 oasc.ZkController.register We are http://127.0.0.1:44347/collection1/ and leader is http://127.0.0.1:44347/collection1/
[junit4:junit4] 2> 8074 T674 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:44347
[junit4:junit4] 2> 8074 T674 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 8075 T674 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 8075 T674 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 8078 T674 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 8080 T631 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 8081 T631 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 8081 T631 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 8158 T631 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 8161 T631 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:52405
[junit4:junit4] 2> 8162 T631 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 8163 T631 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 8164 T631 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1364119272281
[junit4:junit4] 2> 8164 T631 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1364119272281/solr.xml
[junit4:junit4] 2> 8165 T631 oasc.CoreContainer.<init> New CoreContainer 5439095
[junit4:junit4] 2> 8166 T631 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1364119272281/'
[junit4:junit4] 2> 8167 T631 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1364119272281/'
[junit4:junit4] 2> 8190 T631 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 8190 T631 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 8190 T631 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 8191 T631 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 8191 T631 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 8192 T631 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 8192 T631 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 8192 T631 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 8192 T631 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 8193 T631 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 8199 T631 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 8207 T631 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:41151/solr
[junit4:junit4] 2> 8208 T631 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 8208 T631 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:41151 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@1cfe70a
[junit4:junit4] 2> 8209 T631 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 8209 T686 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:41151. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 8210 T686 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:41151, initiating session
[junit4:junit4] 2> 8210 T633 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:44997
[junit4:junit4] 2> 8211 T633 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:44997
[junit4:junit4] 2> 8212 T635 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9bd81bc90007 with negotiated timeout 20000 for client /127.0.0.1:44997
[junit4:junit4] 2> 8212 T686 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:41151, sessionid = 0x13d9bd81bc90007, negotiated timeout = 20000
[junit4:junit4] 2> 8213 T687 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1cfe70a name:ZooKeeperConnection Watcher:127.0.0.1:41151 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 8213 T631 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 8215 T636 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d9bd81bc90007
[junit4:junit4] 2> 8215 T687 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 8215 T633 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:44997 which had sessionid 0x13d9bd81bc90007
[junit4:junit4] 2> 8215 T631 oaz.ZooKeeper.close Session: 0x13d9bd81bc90007 closed
[junit4:junit4] 2> 8216 T631 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 8219 T631 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:41151/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@1a648f0
[junit4:junit4] 2> 8220 T688 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:41151. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 8220 T631 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 8220 T688 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:41151, initiating session
[junit4:junit4] 2> 8221 T633 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:44998
[junit4:junit4] 2> 8221 T633 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:44998
[junit4:junit4] 2> 8222 T635 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9bd81bc90008 with negotiated timeout 20000 for client /127.0.0.1:44998
[junit4:junit4] 2> 8222 T688 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:41151, sessionid = 0x13d9bd81bc90008, negotiated timeout = 20000
[junit4:junit4] 2> 8223 T689 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a648f0 name:ZooKeeperConnection Watcher:127.0.0.1:41151/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 8223 T631 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 8224 T636 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9bd81bc90008 type:create cxid:0x1 zxid:0x6c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 8226 T636 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9bd81bc90008 type:create cxid:0x3 zxid:0x6d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 8229 T631 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 9233 T631 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52405_
[junit4:junit4] 2> 9235 T636 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9bd81bc90008 type:delete cxid:0xd zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:52405_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:52405_
[junit4:junit4] 2> 9237 T631 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52405_
[junit4:junit4] 2> 9241 T653 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 9242 T660 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9242 T673 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9242 T660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9243 T673 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9243 T689 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9245 T653 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 9253 T690 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1364119272281/collection1
[junit4:junit4] 2> 9253 T690 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 9254 T690 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 9255 T690 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 9257 T690 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1364119272281/collection1/'
[junit4:junit4] 2> 9258 T690 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1364119272281/collection1/lib/README' to classloader
[junit4:junit4] 2> 9259 T690 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1364119272281/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 9301 T690 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4] 2> 9339 T690 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 9340 T690 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 9344 T690 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 9535 T654 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 9536 T654 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "node_name":"127.0.0.1:44347_",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:44347__collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:44347"}
[junit4:junit4] 2> 9540 T653 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9540 T673 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9540 T689 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9540 T660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 9680 T690 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 9685 T690 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 9687 T690 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 9707 T690 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 9712 T690 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 9717 T690 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 9720 T690 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 9720 T690 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 9721 T690 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 9723 T690 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 9723 T690 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 9724 T690 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 9724 T690 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1364119272281/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1364119264195/jetty2/
[junit4:junit4] 2> 9725 T690 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@555231
[junit4:junit4] 2> 9725 T690 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 9726 T690 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1364119264195/jetty2
[junit4:junit4] 2> 9727 T690 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1364119264195/jetty2/index/
[junit4:junit4] 2> 9727 T690 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1364119264195/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 9728 T690 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1364119264195/jetty2/index
[junit4:junit4] 2> 9730 T690 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@12e77de lockFactory=org.apache.lucene.store.NativeFSLockFactory@b2abe3),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 9730 T690 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 9734 T690 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 9734 T690 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 9735 T690 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 9736 T690 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 9737 T690 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 9737 T690 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 9738 T690 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 9739 T690 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 9739 T690 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 9744 T690 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 9748 T690 oass.SolrIndexSearcher.<init> Opening Searcher@1694d8d main
[junit4:junit4] 2> 9749 T690 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1364119264195/jetty2/tlog
[junit4:junit4] 2> 9750 T690 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 9750 T690 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 9756 T691 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1694d8d main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 9758 T690 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 9758 T690 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 11046 T654 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 11048 T654 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "node_name":"127.0.0.1:52405_",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "base_url":"http://127.0.0.1:52405"}
[junit4:junit4] 2> 11049 T654 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 11049 T654 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 11055 T653 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 11055 T673 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 11055 T689 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 11055 T660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 11759 T690 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 11760 T690 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:52405 collection:collection1 shard:shard1
[junit4:junit4] 2> 11761 T690 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 11773 T636 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9bd81bc90008 type:delete cxid:0x49 zxid:0x7d txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard1 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard1
[junit4:junit4] 2> 11774 T690 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 11775 T636 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9bd81bc90008 type:create cxid:0x4a zxid:0x7e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 11779 T690 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 11779 T690 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 11779 T690 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:52405/collection1/
[junit4:junit4] 2> 11780 T690 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 11780 T690 oasc.SyncStrategy.syncToMe http://127.0.0.1:52405/collection1/ has no replicas
[junit4:junit4] 2> 11780 T690 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:52405/collection1/
[junit4:junit4] 2> 11781 T690 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 11786 T636 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9bd81bc90008 type:create cxid:0x54 zxid:0x81 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 12563 T654 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 12577 T673 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 12577 T689 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 12577 T660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 12577 T653 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 12593 T690 oasc.ZkController.register We are http://127.0.0.1:52405/collection1/ and leader is http://127.0.0.1:52405/collection1/
[junit4:junit4] 2> 12594 T690 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:52405
[junit4:junit4] 2> 12594 T690 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 12594 T690 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 12594 T690 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 12597 T690 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 12600 T631 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 12601 T631 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 12602 T631 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 12684 T631 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 12687 T631 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:53948
[junit4:junit4] 2> 12687 T631 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 12688 T631 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 12688 T631 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1364119276803
[junit4:junit4] 2> 12688 T631 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1364119276803/solr.xml
[junit4:junit4] 2> 12689 T631 oasc.CoreContainer.<init> New CoreContainer 23475103
[junit4:junit4] 2> 12689 T631 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1364119276803/'
[junit4:junit4] 2> 12690 T631 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1364119276803/'
[junit4:junit4] 2> 12722 T631 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 12722 T631 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 12722 T631 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 12723 T631 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 12723 T631 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 12723 T631 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 12724 T631 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 12724 T631 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 12724 T631 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 12725 T631 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 12730 T631 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 12740 T631 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:41151/solr
[junit4:junit4] 2> 12741 T631 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 12741 T631 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:41151 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@11ab114
[junit4:junit4] 2> 12742 T631 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 12743 T702 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:41151. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 12744 T702 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:41151, initiating session
[junit4:junit4] 2> 12744 T633 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:45008
[junit4:junit4] 2> 12744 T633 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:45008
[junit4:junit4] 2> 12746 T635 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9bd81bc90009 with negotiated timeout 20000 for client /127.0.0.1:45008
[junit4:junit4] 2> 12747 T702 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:41151, sessionid = 0x13d9bd81bc90009, negotiated timeout = 20000
[junit4:junit4] 2> 12747 T703 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@11ab114 name:ZooKeeperConnection Watcher:127.0.0.1:41151 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 12748 T631 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 12750 T636 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d9bd81bc90009
[junit4:junit4] 2> 12750 T703 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 12751 T633 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:45008 which had sessionid 0x13d9bd81bc90009
[junit4:junit4] 2> 12750 T631 oaz.ZooKeeper.close Session: 0x13d9bd81bc90009 closed
[junit4:junit4] 2> 12751 T631 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 12754 T631 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:41151/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@e46cf
[junit4:junit4] 2> 12755 T704 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:41151. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 12755 T631 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 12755 T704 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:41151, initiating session
[junit4:junit4] 2> 12755 T633 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:45009
[junit4:junit4] 2> 12756 T633 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:45009
[junit4:junit4] 2> 12757 T635 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9bd81bc9000a with negotiated timeout 20000 for client /127.0.0.1:45009
[junit4:junit4] 2> 12757 T704 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:41151, sessionid = 0x13d9bd81bc9000a, negotiated timeout = 20000
[junit4:junit4] 2> 12758 T705 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e46cf name:ZooKeeperConnection Watcher:127.0.0.1:41151/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 12758 T631 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 12759 T636 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9bd81bc9000a type:create cxid:0x1 zxid:0x8e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 12761 T636 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9bd81bc9000a type:create cxid:0x3 zxid:0x8f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 12766 T631 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 13771 T631 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53948_
[junit4:junit4] 2> 13772 T636 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9bd81bc9000a type:delete cxid:0xd zxid:0x90 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:53948_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:53948_
[junit4:junit4] 2> 13774 T631 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:53948_
[junit4:junit4] 2> 13778 T673 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 13779 T689 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 13779 T653 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 13780 T705 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 13780 T660 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 13781 T660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 13782 T673 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 13782 T689 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 13783 T653 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 13791 T706 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1364119276803/collection1
[junit4:junit4] 2> 13792 T706 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 13793 T706 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 13793 T706 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 13796 T706 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1364119276803/collection1/'
[junit4:junit4] 2> 13797 T706 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1364119276803/collection1/lib/README' to classloader
[junit4:junit4] 2> 13797 T706 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1364119276803/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 13850 T706 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4] 2> 13880 T706 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 13882 T706 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 13885 T706 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 14087 T654 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 14089 T654 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "node_name":"127.0.0.1:52405_",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:52405__collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:52405"}
[junit4:junit4] 2> 14095 T660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14095 T673 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14095 T689 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14095 T653 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14095 T705 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 14191 T706 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 14196 T706 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 14198 T706 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 14217 T706 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 14223 T706 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 14229 T706 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 14231 T706 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 14232 T706 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 14232 T706 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 14234 T706 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 14235 T706 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 14235 T706 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 14236 T706 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1364119276803/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1364119264195/jetty3/
[junit4:junit4] 2> 14236 T706 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@555231
[junit4:junit4] 2> 14237 T706 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 14238 T706 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1364119264195/jetty3
[junit4:junit4] 2> 14238 T706 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1364119264195/jetty3/index/
[junit4:junit4] 2> 14238 T706 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1364119264195/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 14240 T706 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1364119264195/jetty3/index
[junit4:junit4] 2> 14242 T706 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@88ce88 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1297c83),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 14243 T706 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 14246 T706 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 14246 T706 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 14247 T706 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 14249 T706 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 14250 T706 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 14250 T706 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 14251 T706 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 14252 T706 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 14252 T706 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 14256 T706 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 14258 T706 oass.SolrIndexSearcher.<init> Opening Searcher@314666 main
[junit4:junit4] 2> 14259 T706 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1364119264195/jetty3/tlog
[junit4:junit4] 2> 14259 T706 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 14259 T706 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 14267 T707 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@314666 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 14271 T706 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 14272 T706 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 15602 T654 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 15604 T654 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "node_name":"127.0.0.1:53948_",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "base_url":"http://127.0.0.1:53948"}
[junit4:junit4] 2> 15604 T654 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 15604 T654 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4] 2> 15612 T705 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 15612 T673 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 15612 T653 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 15612 T660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 15612 T689 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 16274 T706 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 16275 T706 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:53948 collection:collection1 shard:shard2
[junit4:junit4] 2> 16281 T706 oasc.ZkController.register We are http://127.0.0.1:53948/collection1/ and leader is http://127.0.0.1:44347/collection1/
[junit4:junit4] 2> 16281 T706 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:53948
[junit4:junit4] 2> 16282 T706 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 16282 T706 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C85 name=collection1 org.apache.solr.core.SolrCore@1d6fbb1 url=http://127.0.0.1:53948/collection1 node=127.0.0.1:53948_ C85_STATE=coll:collection1 core:collection1 props:{state=down, collection=collection1, core=collection1, node_name=127.0.0.1:53948_, base_url=http://127.0.0.1:53948}
[junit4:junit4] 2> 16289 T708 C85 P53948 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 16290 T706 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 16292 T708 C85 P53948 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 16292 T708 C85 P53948 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 16292 T708 C85 P53948 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 16293 T631 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 16294 T631 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 16295 T708 C85 P53948 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 16295 T631 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 16364 T631 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4] 2> 16366 T631 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:59974
[junit4:junit4] 2> 16367 T631 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 16368 T631 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 16368 T631 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1364119280494
[junit4:junit4] 2> 16368 T631 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1364119280494/solr.xml
[junit4:junit4] 2> 16369 T631 oasc.CoreContainer.<init> New CoreContainer 16113410
[junit4:junit4] 2> 16369 T631 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1364119280494/'
[junit4:junit4] 2> 16369 T631 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1364119280494/'
[junit4:junit4] 2> 16395 T631 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 16396 T631 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 16396 T631 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 16397 T631 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 16397 T631 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 16397 T631 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 16398 T631 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 16398 T631 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 16399 T631 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 16399 T631 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 16406 T631 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4] 2> 16416 T631 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:41151/solr
[junit4:junit4] 2> 16417 T631 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 16417 T631 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:41151 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@1994388
[junit4:junit4] 2> 16418 T631 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 16418 T719 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:41151. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 16419 T719 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:41151, initiating session
[junit4:junit4] 2> 16419 T633 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:45017
[junit4:junit4] 2> 16420 T633 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:45017
[junit4:junit4] 2> 16421 T635 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9bd81bc9000b with negotiated timeout 20000 for client /127.0.0.1:45017
[junit4:junit4] 2> 16421 T719 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:41151, sessionid = 0x13d9bd81bc9000b, negotiated timeout = 20000
[junit4:junit4] 2> 16421 T720 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1994388 name:ZooKeeperConnection Watcher:127.0.0.1:41151 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 16422 T631 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 16424 T636 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d9bd81bc9000b
[junit4:junit4] 2> 16425 T720 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 16425 T633 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:45017 which had sessionid 0x13d9bd81bc9000b
[junit4:junit4] 2> 16425 T631 oaz.ZooKeeper.close Session: 0x13d9bd81bc9000b closed
[junit4:junit4] 2> 16425 T631 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 16428 T631 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:41151/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@1a15946
[junit4:junit4] 2> 16429 T721 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:41151. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 16429 T631 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 16429 T721 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:41151, initiating session
[junit4:junit4] 2> 16430 T633 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:45018
[junit4:junit4] 2> 16430 T633 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:45018
[junit4:junit4] 2> 16431 T635 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9bd81bc9000c with negotiated timeout 20000 for client /127.0.0.1:45018
[junit4:junit4] 2> 16431 T721 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:41151, sessionid = 0x13d9bd81bc9000c, negotiated timeout = 20000
[junit4:junit4] 2> 16431 T722 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a15946 name:ZooKeeperConnection Watcher:127.0.0.1:41151/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 16432 T631 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 16433 T636 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9bd81bc9000c type:create cxid:0x1 zxid:0xa1 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 16435 T636 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9bd81bc9000c type:create cxid:0x3 zxid:0xa2 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 16438 T631 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 17120 T654 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 17122 T654 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "node_name":"127.0.0.1:53948_",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:53948__collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:53948"}
[junit4:junit4] 2> 17131 T653 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17131 T673 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17131 T705 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17131 T660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17131 T689 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17132 T722 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17311 T667 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {wt=javabin&action=PREPRECOVERY&state=recovering&onlyIfLeader=true&coreNodeName=127.0.0.1:53948__collection1&checkLive=true&core=collection1&version=2&nodeName=127.0.0.1:53948_} status=0 QTime=1003
[junit4:junit4] 2> 17442 T631 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59974_
[junit4:junit4] 2> 17443 T636 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9bd81bc9000c type:delete cxid:0xf zxid:0xa7 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:59974_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:59974_
[junit4:junit4] 2> 17444 T631 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:59974_
[junit4:junit4] 2> 17449 T722 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 17449 T673 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 17449 T689 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 17450 T653 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 17450 T673 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 17450 T705 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 17449 T660 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 17450 T653 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 17451 T689 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 17451 T705 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 17451 T660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 17455 T722 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 17462 T723 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1364119280494/collection1
[junit4:junit4] 2> 17462 T723 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 17464 T723 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 17464 T723 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 17466 T723 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1364119280494/collection1/'
[junit4:junit4] 2> 17467 T723 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1364119280494/collection1/lib/README' to classloader
[junit4:junit4] 2> 17468 T723 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1364119280494/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 17506 T723 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4] 2> 17537 T723 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 17538 T723 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 17542 T723 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> 17868 T723 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 17874 T723 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 17877 T723 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 17896 T723 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 17901 T723 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 17907 T723 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 17908 T723 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 17909 T723 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 17910 T723 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 17911 T723 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 17912 T723 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 17912 T723 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 17913 T723 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1364119280494/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1364119264195/jetty4/
[junit4:junit4] 2> 17913 T723 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@555231
[junit4:junit4] 2> 17914 T723 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 17915 T723 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1364119264195/jetty4
[junit4:junit4] 2> 17915 T723 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1364119264195/jetty4/index/
[junit4:junit4] 2> 17915 T723 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1364119264195/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 17916 T723 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1364119264195/jetty4/index
[junit4:junit4] 2> 17918 T723 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@118dd2a lockFactory=org.apache.lucene.store.NativeFSLockFactory@92a725),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 17919 T723 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 17922 T723 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 17922 T723 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 17923 T723 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 17924 T723 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 17925 T723 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 17925 T723 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 17926 T723 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 17927 T723 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 17928 T723 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 17932 T723 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 17936 T723 oass.SolrIndexSearcher.<init> Opening Searcher@eb8d1b main
[junit4:junit4] 2> 17936 T723 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1364119264195/jetty4/tlog
[junit4:junit4] 2> 17937 T723 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 17937 T723 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 17941 T724 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@eb8d1b main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 17943 T723 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 17943 T723 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 18641 T654 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 18642 T654 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "node_name":"127.0.0.1:59974_",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "base_url":"http://127.0.0.1:59974"}
[junit4:junit4] 2> 18642 T654 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 18642 T654 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 18649 T722 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 18649 T660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 18649 T673 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 18649 T653 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 18649 T689 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 18649 T705 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 18945 T723 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 18945 T723 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:59974 collection:collection1 shard:shard1
[junit4:junit4] 2> 18951 T723 oasc.ZkController.register We are http://127.0.0.1:59974/collection1/ and leader is http://127.0.0.1:52405/collection1/
[junit4:junit4] 2> 18952 T723 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:59974
[junit4:junit4] 2> 18952 T723 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 18952 T723 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C86 name=collection1 org.apache.solr.core.SolrCore@171cf46 url=http://127.0.0.1:59974/collection1 node=127.0.0.1:59974_ C86_STATE=coll:collection1 core:collection1 props:{state=down, collection=collection1, core=collection1, node_name=127.0.0.1:59974_, base_url=http://127.0.0.1:59974}
[junit4:junit4] 2> 18953 T725 C86 P59974 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 18954 T725 C86 P59974 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 18955 T723 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 18955 T725 C86 P59974 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 18955 T725 C86 P59974 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 18957 T631 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 18957 T725 C86 P59974 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 18958 T631 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 18959 T631 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 18970 T631 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 18972 T631 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 18982 T666 oasc.CoreContainer.create Creating SolrCore 'onenodecollectioncore' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1364119267751/onenodecollectioncore
[junit4:junit4] 2> 18983 T666 oasc.ZkController.createCollectionZkNode Check for collection zkNode:onenodecollection
[junit4:junit4] 2> 18984 T666 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:onenodecollection
[junit4:junit4] 2> 18985 T666 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 18988 T666 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 18988 T666 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection
[junit4:junit4] 2> 18992 T666 oasc.ZkController.readConfigName Load collection config from:/collections/onenodecollection
[junit4:junit4] 2> 18995 T666 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1364119267751/onenodecollectioncore/'
[junit4:junit4] 2> 19034 T666 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_43
[junit4:junit4] 2> 19062 T666 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 19064 T666 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4] 2> 19067 T666 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4] 2> C85_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=recovering, collection=collection1, core=collection1, node_name=127.0.0.1:53948_, base_url=http://127.0.0.1:53948}
[junit4:junit4] 2> 19313 T708 C85 P53948 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:44347/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 19317 T708 C85 P53948 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4] 2> 19321 T708 C85 P53948 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:53948 START replicas=[http://127.0.0.1:44347/collection1/] nUpdates=100
[junit4:junit4] 2> 19322 T708 C85 P53948 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 19323 T708 C85 P53948 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 19323 T708 C85 P53948 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 19323 T708 C85 P53948 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 19323 T708 C85 P53948 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 19323 T708 C85 P53948 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:44347/collection1/. core=collection1
[junit4:junit4] 2> 19324 T708 C85 P53948 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C87 name=collection1 org.apache.solr.core.SolrCore@625912 url=http://127.0.0.1:44347/collection1 node=127.0.0.1:44347_ C87_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, collection=collection1, core=collection1, node_name=127.0.0.1:44347_, base_url=http://127.0.0.1:44347, leader=true}
[junit4:junit4] 2> 19328 T668 C87 P44347 REQ /get {qt=/get&wt=javabin&distrib=false&getVersions=100&version=2} status=0 QTime=0
[junit4:junit4] 2> 19330 T667 C87 P44347 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 19331 T667 C87 P44347 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@132412 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1bd372f),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 19332 T667 C87 P44347 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 19332 T667 C87 P44347 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@132412 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1bd372f),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@132412 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1bd372f),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> 19332 T667 C87 P44347 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4] 2> 19333 T667 C87 P44347 oass.SolrIndexSearcher.<init> Opening Searcher@1b50818 realtime
[junit4:junit4] 2> 19333 T667 C87 P44347 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 19333 T667 C87 P44347 /update {softCommit=false&wt=javabin&commit_end_point=true&waitSearcher=true&version=2&openSearcher=false&commit=true} {commit=} 0 4
[junit4:junit4] 2> 19362 T708 C85 P53948 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 19362 T708 C85 P53948 oascsi.HttpClientUtil.createClient Creating new http client, config:connTimeout=5000&socketTimeout=20000&allowCompression=false&maxConnections=10000&maxConnectionsPerHost=10000
[junit4:junit4] 2> 19367 T708 C85 P53948 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 19370 T668 C87 P44347 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 19371 T668 C87 P44347 REQ /replication {qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=2
[junit4:junit4] 2> 19372 T708 C85 P53948 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 19372 T708 C85 P53948 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 19372 T708 C85 P53948 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 19374 T667 C87 P44347 REQ /replication {qt=/replication&wt=javabin&generation=2&version=2&command=filelist} status=0 QTime=1
[junit4:junit4] 2> 19374 T708 C85 P53948 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 19375 T708 C85 P53948 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1364119264195/jetty3/index.20130324100123567
[junit4:junit4] 2> 19375 T708 C85 P53948 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@108b34f lockFactory=org.apache.lucene.store.NativeFSLockFactory@5356e2) fullCopy=false
[junit4:junit4] 2> 19381 T669 C87 P44347 REQ /replication {checksum=true&qt=/replication&wt=filestream&file=segments_2&generation=2&command=filecontent} status=0 QTime=1
[junit4:junit4] 2> 19385 T708 C85 P53948 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 19387 T708 C85 P53948 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 19387 T708 C85 P53948 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 19388 T708 C85 P53948 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@88ce88 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1297c83),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@88ce88 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1297c83),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> 19388 T708 C85 P53948 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4] 2> 19388 T708 C85 P53948 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 19389 T708 C85 P53948 oass.SolrIndexSearcher.<init> Opening Searcher@71d185 main
[junit4:junit4] 2> 19389 T707 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@71d185 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 19390 T708 C85 P53948 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 19390 T708 C85 P53948 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 19390 T708 C85 P53948 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 19390 T708 C85 P53948 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 19392 T708 C85 P53948 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 19400 T666 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 19405 T666 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 19408 T666 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 19423 T666 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 19429 T666 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 19433 T666 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 19435 T666 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 19435 T666 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 19436 T666 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 19438 T666 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 19439 T666 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARNING Expected key, got STRING
[junit4:junit4] 2> 19439 T666 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 19440 T666 oasc.SolrCore.<init> [onenodecollectioncore] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1364119267751/onenodecollectioncore/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1364119264196/onenodecollection/
[junit4:junit4] 2> 19440 T666 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@555231
[junit4:junit4] 2> 19441 T666 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 19442 T666 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1364119264196/onenodecollection
[junit4:junit4] 2> 19443 T666 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1364119264196/onenodecollection/index/
[junit4:junit4] 2> 19444 T666 oasc.SolrCore.initIndex WARNING [onenodecollectioncore] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1364119264196/onenodecollection/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 19444 T666 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1364119264196/onenodecollection/index
[junit4:junit4] 2> 19447 T666 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@11f8b5c lockFactory=org.apache.lucene.store.NativeFSLockFactory@140e198),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 19447 T666 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 19450 T666 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 19451 T666 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 19452 T666 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 19453 T666 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 19454 T666 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 19454 T666 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 19455 T666 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 19456 T666 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 19456 T666 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 19461 T666 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 19465 T666 oass.SolrIndexSearcher.<init> Opening Searcher@18cde19 main
[junit4:junit4] 2> 19466 T666 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1364119264196/onenodecollection/tlog
[junit4:junit4] 2> 19467 T666 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 19467 T666 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 19473 T729 oasc.SolrCore.registerSearcher [onenodecollectioncore] Registered new searcher Searcher@18cde19 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 19475 T666 oasc.ZkController.publish publishing core=onenodecollectioncore state=down
[junit4:junit4] 2> 20157 T654 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 20159 T654 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "node_name":"127.0.0.1:59974_",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:59974__collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:59974"}
[junit4:junit4] 2> 20166 T654 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "node_name":"127.0.0.1:53948_",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:53948__collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:53948"}
[junit4:junit4] 2> 20173 T654 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "core":"onenodecollectioncore",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "node_name":"127.0.0.1:44347_",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "base_url":"http://127.0.0.1:44347"}
[junit4:junit4] 2> 20173 T654 oasc.Overseer$ClusterStateUpdater.createCollection Create collection onenodecollection with numShards 1
[junit4:junit4] 2> 20174 T654 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 20182 T705 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 20183 T689 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 20183 T660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 20182 T722 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 20184 T653 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 20182 T673 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 20478 T666 oasc.CoreContainer.registerCore registering core: onenodecollectioncore
[junit4:junit4] 2> 20479 T666 oasc.ZkController.register Register replica - core:onenodecollectioncore address:http://127.0.0.1:44347 collection:onenodecollection shard:shard1
[junit4:junit4] 2> 20480 T666 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection/leader_elect/shard1/election
[junit4:junit4] 2> 20494 T636 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9bd81bc90006 type:delete cxid:0xac zxid:0xc2 txntype:-1 reqpath:n/a Error Path:/solr/collections/onenodecollection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/onenodecollection/leaders
[junit4:junit4] 2> 20495 T666 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 20496 T636 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9bd81bc90006 type:create cxid:0xad zxid:0xc3 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 20500 T666 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 20501 T666 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 20501 T666 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:44347/onenodecollectioncore/
[junit4:junit4] 2> 20502 T666 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 20502 T666 oasc.SyncStrategy.syncToMe http://127.0.0.1:44347/onenodecollectioncore/ has no replicas
[junit4:junit4] 2> 20503 T666 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:44347/onenodecollectioncore/
[junit4:junit4] 2> 20503 T666 oascc.SolrZkClient.makePath makePath: /collections/onenodecollection/leaders/shard1
[junit4:junit4] 2> 20512 T636 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9bd81bc90006 type:create cxid:0xb8 zxid:0xc7 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 20972 T684 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {wt=javabin&action=PREPRECOVERY&state=recovering&onlyIfLeader=true&coreNodeName=127.0.0.1:59974__collection1&checkLive=true&core=collection1&version=2&nodeName=127.0.0.1:59974_} status=0 QTime=2001
[junit4:junit4] 2> 21698 T654 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 21713 T673 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 21713 T689 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 21713 T653 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 21714 T660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 21713 T705 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 21713 T722 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 21721 T666 oasc.ZkController.register We are http://127.0.0.1:44347/onenodecollectioncore/ and leader is http://127.0.0.1:44347/onenodecollectioncore/
[junit4:junit4] 2> 21722 T666 oasc.ZkController.register No LogReplay needed for core=onenodecollectioncore baseURL=http://127.0.0.1:44347
[junit4:junit4] 2> 21722 T666 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 21723 T666 oasc.ZkController.publish publishing core=onenodecollectioncore state=active
[junit4:junit4] 2> 21726 T666 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 21728 T666 oass.SolrDispatchFilter.handleAdminRequest /admin/cores {wt=javabin&dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1364119264196/onenodecollection&action=CREATE&roles=none&collection=onenodecollection&name=onenodecollectioncore&version=2&numShards=1} status=0 QTime=2746
[junit4:junit4] 2> 21729 T631 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: onenodecollection failOnTimeout:true timeout (sec):230
[junit4:junit4] 2> 21731 T631 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 22734 T631 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C86_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, collection=collection1, core=collection1, node_name=127.0.0.1:59974_, base_url=http://127.0.0.1:59974}
[junit4:junit4] 2> 22973 T725 C86 P59974 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:52405/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 22974 T725 C86 P59974 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:59974 START replicas=[http://127.0.0.1:52405/collection1/] nUpdates=100
[junit4:junit4] 2> 22974 T725 C86 P59974 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 22975 T725 C86 P59974 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 22975 T725 C86 P59974 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 22975 T725 C86 P59974 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 22975 T725 C86 P59974 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 22977 T725 C86 P59974 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:52405/collection1/. core=collection1
[junit4:junit4] 2> 22977 T725 C86 P59974 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C88 name=collection1 org.apache.solr.core.SolrCore@13fdf2e url=http://127.0.0.1:52405/collection1 node=127.0.0.1:52405_ C88_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, collection=collection1, core=collection1, node_name=127.0.0.1:52405_, base_url=http://127.0.0.1:52405, leader=true}
[junit4:junit4] 2> 22985 T684 C88 P52405 REQ /get {qt=/get&wt=javabin&distrib=false&getVersions=100&version=2} status=0 QTime=1
[junit4:junit4] 2> 22988 T682 C88 P52405 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 22990 T682 C88 P52405 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@12e77de lockFactory=org.apache.lucene.store.NativeFSLockFactory@b2abe3),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> 22992 T682 C88 P52405 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4] 2> 22993 T682 C88 P52405 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@12e77de lockFactory=org.apache.lucene.store.NativeFSLockFactory@b2abe3),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@12e77de lockFactory=org.apache.lucene.store.NativeFSLockFactory@b2abe3),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> 22993 T682 C88 P52405 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4] 2> 22994 T682 C88 P52405 oass.SolrIndexSearcher.<init> Opening Searcher@13565f2 realtime
[junit4:junit4] 2> 22995 T682 C88 P52405 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 22996 T682 C88 P52405 /update {softCommit=false&wt=javabin&commit_end_point=true&waitSearcher=true&version=2&openSearcher=false&commit=true} {commit=} 0 8
[junit4:junit4] 2> 22997 T725 C86 P59974 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 22998 T725 C86 P59974 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 23001 T684 C88 P52405 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 23002 T684 C88 P52405 REQ /replication {qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=2
[junit4:junit4] 2> 23003 T725 C86 P59974 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 23003 T725 C86 P59974 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 23004 T725 C86 P59974 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 23006 T685 C88 P52405 REQ /replication {qt=/replication&wt=javabin&generation=2&version=2&command=filelist} status=0 QTime=0
[junit4:junit4] 2> 23007 T725 C86 P59974 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 23009 T725 C86 P59974 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1364119264195/jetty4/index.20130324100127200
[junit4:junit4] 2> 23009 T725 C86 P59974 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@1daf503 lockFactory=org.apache.lucene.store.NativeFSLockFactory@38fd9e) fullCopy=false
[junit4:junit4] 2> 23013 T682 C88 P52405 REQ /replication {checksum=true&qt=/replication&wt=filestream&file=segments_2&generation=2&command=filecontent} status=0 QTime=0
[junit4:junit4] 2> 23016 T725 C86 P59974 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 23017 T725 C86 P59974 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 23017 T725 C86 P59974 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 23019 T725 C86 P59974 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@118dd2a lockFactory=org.apache.lucene.store.NativeFSLockFactory@92a725),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@118dd2a lockFactory=org.apache.lucene.store.NativeFSLockFactory@92a725),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 2> 23020 T725 C86 P59974 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4] 2> 23021 T725 C86 P59974 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 23021 T725 C86 P59974 oass.SolrIndexSearcher.<init> Opening Searcher@1ab06e3 main
[junit4:junit4] 2> 23023 T724 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1ab06e3 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 23024 T725 C86 P59974 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 23024 T725 C86 P59974 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 23024 T725 C86 P59974 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 23024 T725 C86 P59974 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 23026 T725 C86 P59974 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 23225 T654 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 23227 T654 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "core":"onenodecollectioncore",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "node_name":"127.0.0.1:44347_",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:44347__onenodecollectioncore",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:44347"}
[junit4:junit4] 2> 23234 T654 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "node_name":"127.0.0.1:59974_",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:59974__collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:59974"}
[junit4:junit4] 2> 23242 T660 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 23242 T689 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 23242 T722 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 23242 T705 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 23242 T673 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 23242 T653 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 23736 T631 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 23738 T631 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: onenodecollection
[junit4:junit4] 2> 23739 T631 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 23759 T631 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:41151 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@ed4249
[junit4:junit4] 2> 23761 T732 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost.localdomain/127.0.0.1:41151. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4] 2> 23761 T631 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 23761 T732 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost.localdomain/127.0.0.1:41151, initiating session
[junit4:junit4] 2> 23762 T633 oazs.NIOServerCnxnFactory.run Accepted socket connection from /127.0.0.1:45049
[junit4:junit4] 2> 23762 T633 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /127.0.0.1:45049
[junit4:junit4] 2> 23764 T635 oazs.ZooKeeperServer.finishSessionInit Established session 0x13d9bd81bc9000d with negotiated timeout 10000 for client /127.0.0.1:45049
[junit4:junit4] 2> 23764 T732 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost.localdomain/127.0.0.1:41151, sessionid = 0x13d9bd81bc9000d, negotiated timeout = 10000
[junit4:junit4] 2> 23764 T733 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@ed4249 name:ZooKeeperConnection Watcher:127.0.0.1:41151 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 23765 T631 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 23858 T636 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d9bd81bc9000d
[junit4:junit4] 2> 23859 T733 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 23859 T633 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:45049 which had sessionid 0x13d9bd81bc9000d
[junit4:junit4] 2> 23859 T631 oaz.ZooKeeper.close Session: 0x13d9bd81bc9000d closed
[junit4:junit4] 2> 23860 T636 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d9bd81bc90004
[junit4:junit4] 2> 23861 T633 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:44985 which had sessionid 0x13d9bd81bc90004
[junit4:junit4] 2> 23861 T660 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 23861 T631 oaz.ZooKeeper.close Session: 0x13d9bd81bc90004 closed
[junit4:junit4] 2> 23867 T631 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 60538
[junit4:junit4] 2> 23867 T631 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=4602512
[junit4:junit4] 2> 24754 T654 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 24755 T654 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "node_name":"127.0.0.1:60538_",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:60538__collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:60538"}
[junit4:junit4] 2> 24763 T689 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 24763 T673 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 24764 T705 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 24763 T722 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 24763 T653 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 25870 T631 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 25871 T631 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 25875 T631 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@9cb0d9
[junit4:junit4] 2> 25884 T631 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 25885 T631 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 25885 T631 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 25887 T631 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 25890 T631 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 25891 T631 oasc.CachingDirectoryFactory.close Closing directory when closing factory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1364119264195/control/data
[junit4:junit4] 2> 25893 T631 oasc.CachingDirectoryFactory.closeDirectory Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1364119264195/control/data
[junit4:junit4] 2> 25894 T631 oasc.CachingDirectoryFactory.close Closing directory when closing factory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1364119264195/control/data/index
[junit4:junit4] 2> 25895 T631 oasc.CachingDirectoryFactory.closeDirectory Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1364119264195/control/data/index
[junit4:junit4] 2> 25897 T654 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89398920098676739-127.0.0.1:60538_-n_0000000000) am no longer a leader.
[junit4:junit4] 2> 25898 T636 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d9bd81bc90003
[junit4:junit4] 2> 25898 T705 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 25898 T689 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 25898 T722 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 25899 T653 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 25899 T633 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:44977 which had sessionid 0x13d9bd81bc90003
[junit4:junit4] 2> 25899 T631 oaz.ZooKeeper.close Session: 0x13d9bd81bc90003 closed
[junit4:junit4] 2> 25899 T653 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 25900 T653 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 25900 T653 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> 25900 T673 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 25900 T673 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 25901 T722 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 25901 T689 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 25901 T705 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 25902 T636 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9bd81bc90006 type:delete cxid:0xc9 zxid:0xe3 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4] 2> 25902 T673 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 25904 T673 oasc.Overseer.start Overseer (id=89398920098676742-127.0.0.1:44347_-n_0000000001) starting
[junit4:junit4] 2> 25904 T636 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9bd81bc90006 type:create cxid:0xce zxid:0xe5 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 25905 T636 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9bd81bc90006 type:create cxid:0xcf zxid:0xe6 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 25906 T636 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9bd81bc90006 type:create cxid:0xd1 zxid:0xe7 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 25908 T636 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13d9bd81bc90006 type:create cxid:0xd3 zxid:0xe8 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4] 2> 25910 T735 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 25911 T734 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 25913 T734 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 25913 T734 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "node_name":"127.0.0.1:60538_",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:60538__collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:60538"}
[junit4:junit4] 2> 25921 T631 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
[junit4:junit4] 2> 25923 T705 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 25923 T689 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 25923 T722 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 25923 T673 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 25973 T631 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 44347
[junit4:junit4] 2> 25974 T631 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=11485985
[junit4:junit4] 2> 27429 T734 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 27431 T734 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "node_name":"127.0.0.1:44347_",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:44347__collection1",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:44347"}
[junit4:junit4] 2> 27438 T734 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "collection":"onenodecollection",
[junit4:junit4] 2> "roles":"none",
[junit4:junit4] 2> "core":"onenodecollectioncore",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "node_name":"127.0.0.1:44347_",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:44347__onenodecollectioncore",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:44347"}
[junit4:junit4] 2> 27446 T705 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 27446 T689 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 27446 T673 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 27446 T722 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 28979 T631 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 28980 T631 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 28984 T631 oasc.ZkController.publish publishing core=onenodecollectioncore state=down
[junit4:junit4] 2> 28987 T631 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@625912
[junit4:junit4] 2> 28993 T631 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 28994 T631 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 28995 T631 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 28995 T631 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 28998 T631 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 28998 T631 oasc.CachingDirectoryFactory.close Closing directory when closing factory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1364119264195/jetty1/index
[junit4:junit4] 2> 28999 T631 oasc.CachingDirectoryFactory.closeDirectory Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1364119264195/jetty1/index
[junit4:junit4] 2> 28999 T631 oasc.CachingDirectoryFactory.close Closing directory when closing factory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1364119264195/jetty1
[junit4:junit4] 2> 28999 T631 oasc.CachingDirectoryFactory.closeDirectory Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1364119264195/jetty1
[junit4:junit4] 2> 29000 T631 oasc.SolrCore.close [onenodecollectioncore] CLOSING SolrCore org.apache.solr.core.SolrCore@14fdac3
[junit4:junit4] 2> 29004 T631 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 2> 29004 T631 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 29004 T631 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 29004 T631 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> 29005 T631 oasc.SolrCore.closeSearcher [onenodecollectioncore] Closing main searcher on request.
[junit4:junit4] 2> 29006 T631 oasc.CachingDirectoryFactory.close Closing directory when closing factory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1364119264196/onenodecollection/index
[junit4:junit4] 2> 29006 T631 oasc.CachingDirectoryFactory.closeDirectory Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1364119264196/onenodecollection/index
[junit4:junit4] 2> 29007 T631 oasc.CachingDirectoryFactory.close Closing directory when closing factory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1364119264196/onenodecollection
[junit4:junit4] 2> 29007 T631 oasc.CachingDirectoryFactory.closeDirectory Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1364119264196/onenodecollection
[junit4:junit4] 2> 29008 T734 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89398920098676742-127.0.0.1:44347_-n_0000000001) am no longer a leader.
[junit4:junit4] 2> 29009 T636 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13d9bd81bc90006
[junit4:junit4] 2> 29010 T633 oazs.NIOServerCnxn.closeSock Closed socket connection for client /127.0.0.1:44987 which had sessionid 0x13d9bd81bc90006
[junit4:junit4] 2> 29010 T631 oaz.ZooKeeper.close Session: 0x13d9bd81bc90006 closed
[junit4:junit4] 2> 29010 T636 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when pro
[...truncated too long message...]
1> /solr/collections/control_collection (1)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/control_collection/shards (0)
[junit4:junit4] 1>
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=UnloadDistributedZkTest -Dtests.method=testDistribSearch -Dtests.seed=9EA55DA4301C2E2D -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ro_RO -Dtests.timezone=Poland -Dtests.file.encoding=US-ASCII
[junit4:junit4] FAILURE 137s J1 | UnloadDistributedZkTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: Still found shard2 in collection test_unload_shard_and_collection
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([9EA55DA4301C2E2D:1F43D3BC47434E11]:0)
[junit4:junit4] > at org.junit.Assert.fail(Assert.java:93)
[junit4:junit4] > at org.apache.solr.cloud.UnloadDistributedZkTest.testUnloadShardAndCollection(UnloadDistributedZkTest.java:127)
[junit4:junit4] > at org.apache.solr.cloud.UnloadDistributedZkTest.doTest(UnloadDistributedZkTest.java:77)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
[junit4:junit4] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4:junit4] > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[junit4:junit4] > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4:junit4] > at java.lang.reflect.Method.invoke(Method.java:487)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4] > at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4:junit4] > at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4:junit4] > at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4] > at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
[junit4:junit4] > at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4] > at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4] > at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4] > at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4] > at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4] > at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4] > at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4] > at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2> 137313 T1953 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 137322 T1952 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 3 leaked thread(s).
[junit4:junit4] 2> 137346 T1956 oazs.SessionTrackerImpl.run SessionTrackerImpl exited loop!
[junit4:junit4] 2> 138407 T2035 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 138407 T2035 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 138407 T2035 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 138408 T2035 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4] 2> NOTE: test params are: codec=Appending, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=ro_RO, timezone=Poland
[junit4:junit4] 2> NOTE: Linux 3.2.0-39-generic i386/Oracle Corporation 1.8.0-ea (32-bit)/cpus=8,threads=1,free=147457528,total=320864256
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TestDynamicFieldResource, SampleTest, TestDistributedGrouping, URLClassifyProcessorTest, TestMergePolicyConfig, DocumentAnalysisRequestHandlerTest, BasicZkTest, SortByFunctionTest, TestDefaultSimilarityFactory, SolrIndexConfigTest, MultiTermTest, TestRTGBase, TestRangeQuery, TestLuceneMatchVersion, TestLMJelinekMercerSimilarityFactory, TestFiltering, DisMaxRequestHandlerTest, IndexSchemaRuntimeFieldTest, DebugComponentTest, TestCollationField, TestMultiCoreConfBootstrap, ShardRoutingCustomTest, SolrCoreCheckLockOnStartupTest, HighlighterConfigTest, TestReload, SuggesterTSTTest, TestArbitraryIndexDir, TestZkChroot, CachingDirectoryFactoryTest, TestCSVResponseWriter, ZkSolrClientTest, TestOmitPositions, SpellPossibilityIteratorTest, TestJmxMonitoredMap, TestDynamicFieldCollectionResource, EchoParamsTest, TestFoldingMultitermQuery, TestHashPartitioner, SignatureUpdateProcessorFactoryTest, CopyFieldTest, ConvertedLegacyTest, TestWordDelimiterFilterFactory, HardAutoCommitTest, TestCSVLoader, TestSolr4Spatial, TestCopyFieldCollectionResource, TestPropInject, DOMUtilTest, BasicDistributedZk2Test, UpdateParamsTest, OpenCloseCoreStressTest, DistanceFunctionTest, TestCodecSupport, TestAtomicUpdateErrorCases, OutputWriterTest, TestStressLucene, DateMathParserTest, TestSolrDiscoveryProperties, TestQueryTypes, ClusterStateTest, TestNumberUtils, HighlighterTest, TestBM25SimilarityFactory, TestPostingsSolrHighlighter, ChaosMonkeySafeLeaderTest, UnloadDistributedZkTest]
[junit4:junit4] Completed on J1 in 138.60s, 1 test, 1 failure <<< FAILURES!
[...truncated 417 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:381: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:361: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:183: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:449: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1212: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:876: There were test failures: 273 suites, 1149 tests, 1 error, 1 failure, 13 ignored (7 assumptions)
Total time: 41 minutes 43 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Description set: Java: 32bit/jdk1.8.0-ea-b82 -server -XX:+UseG1GC -XX:MarkStackSize=256K
Email was triggered for: Failure
Sending email for trigger: Failure