You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2013/01/31 10:48:25 UTC

[JENKINS] Lucene-Solr-Tests-trunk-java7 - Build # 3695 - Still Failing

Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-java7/3695/

3 tests failed.
REGRESSION:  org.apache.solr.cloud.SyncSliceTest.testDistribSearch

Error Message:
shard1 should have just been set up to be inconsistent - but it's still consistent

Stack Trace:
java.lang.AssertionError: shard1 should have just been set up to be inconsistent - but it's still consistent
	at __randomizedtesting.SeedInfo.seed([4B69744A3D9F79F0:CA8FFA524AC019CC]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNotNull(Assert.java:526)
	at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:214)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:794)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:722)


FAILED:  org.apache.solr.cloud.BasicDistributedZkTest.testDistribSearch

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([4B69744A3D9F79F0]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([4B69744A3D9F79F0]:0)




Build Log:
[...truncated 9265 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.SyncSliceTest
[junit4:junit4]   2> 0 T3417 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /d_tj/
[junit4:junit4]   2> 4 T3417 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./solrtest-SyncSliceTest-1359619115501
[junit4:junit4]   2> 6 T3417 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 7 T3418 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 10 T3418 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 10 T3418 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4]   2> 10 T3418 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4]   2> 11 T3418 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 11 T3418 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4]   2> 107 T3417 oasc.ZkTestServer.run start zk server on port:54841
[junit4:junit4]   2> 107 T3417 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54841 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@77bd2e5f
[junit4:junit4]   2> 108 T3417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 108 T3423 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:54841. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 109 T3423 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:54841, initiating session
[junit4:junit4]   2> 109 T3419 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:54842
[junit4:junit4]   2> 110 T3419 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:54842
[junit4:junit4]   2> 110 T3421 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 113 T3421 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c8f9d49f40000 with negotiated timeout 10000 for client /140.211.11.196:54842
[junit4:junit4]   2> 113 T3423 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:54841, sessionid = 0x13c8f9d49f40000, negotiated timeout = 10000
[junit4:junit4]   2> 113 T3424 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@77bd2e5f name:ZooKeeperConnection Watcher:127.0.0.1:54841 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 113 T3417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 114 T3417 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 117 T3422 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c8f9d49f40000
[junit4:junit4]   2> 131 T3424 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 131 T3419 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:54842 which had sessionid 0x13c8f9d49f40000
[junit4:junit4]   2> 131 T3417 oaz.ZooKeeper.close Session: 0x13c8f9d49f40000 closed
[junit4:junit4]   2> 132 T3417 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54841/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@42c72ea0
[junit4:junit4]   2> 132 T3417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 133 T3425 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:54841. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 133 T3425 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:54841, initiating session
[junit4:junit4]   2> 133 T3419 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:54843
[junit4:junit4]   2> 134 T3419 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:54843
[junit4:junit4]   2> 135 T3421 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c8f9d49f40001 with negotiated timeout 10000 for client /140.211.11.196:54843
[junit4:junit4]   2> 135 T3425 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:54841, sessionid = 0x13c8f9d49f40001, negotiated timeout = 10000
[junit4:junit4]   2> 136 T3426 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@42c72ea0 name:ZooKeeperConnection Watcher:127.0.0.1:54841/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 136 T3417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 136 T3417 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 140 T3417 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 142 T3417 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 145 T3417 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 148 T3417 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 149 T3417 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 154 T3417 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 155 T3417 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 258 T3417 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 259 T3417 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 262 T3417 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 263 T3417 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 266 T3417 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 267 T3417 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 274 T3417 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 275 T3417 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 278 T3417 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 279 T3417 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 281 T3417 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 282 T3417 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 285 T3417 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 286 T3417 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 293 T3422 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c8f9d49f40001
[junit4:junit4]   2> 294 T3426 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 294 T3419 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:54843 which had sessionid 0x13c8f9d49f40001
[junit4:junit4]   2> 294 T3417 oaz.ZooKeeper.close Session: 0x13c8f9d49f40001 closed
[junit4:junit4]   2> 457 T3417 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 462 T3417 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:54844
[junit4:junit4]   2> 463 T3417 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 464 T3417 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 464 T3417 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1359619115791
[junit4:junit4]   2> 465 T3417 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1359619115791/solr.xml
[junit4:junit4]   2> 465 T3417 oasc.CoreContainer.<init> New CoreContainer 1413950341
[junit4:junit4]   2> 466 T3417 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1359619115791/'
[junit4:junit4]   2> 466 T3417 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1359619115791/'
[junit4:junit4]   2> 523 T3417 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 524 T3417 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 524 T3417 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 525 T3417 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 525 T3417 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 526 T3417 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 526 T3417 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 527 T3417 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 527 T3417 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 528 T3417 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 550 T3417 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 570 T3417 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:54841/solr
[junit4:junit4]   2> 570 T3417 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 571 T3417 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54841 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@258bfca
[junit4:junit4]   2> 572 T3417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 573 T3436 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:54841. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 573 T3436 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:54841, initiating session
[junit4:junit4]   2> 573 T3419 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:54845
[junit4:junit4]   2> 574 T3419 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:54845
[junit4:junit4]   2> 576 T3421 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c8f9d49f40002 with negotiated timeout 20000 for client /140.211.11.196:54845
[junit4:junit4]   2> 576 T3436 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:54841, sessionid = 0x13c8f9d49f40002, negotiated timeout = 20000
[junit4:junit4]   2> 576 T3437 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@258bfca name:ZooKeeperConnection Watcher:127.0.0.1:54841 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 576 T3417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 577 T3422 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c8f9d49f40002
[junit4:junit4]   2> 578 T3437 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 578 T3419 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:54845 which had sessionid 0x13c8f9d49f40002
[junit4:junit4]   2> 578 T3417 oaz.ZooKeeper.close Session: 0x13c8f9d49f40002 closed
[junit4:junit4]   2> 579 T3417 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 591 T3417 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54841/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@5d4a5bd9
[junit4:junit4]   2> 592 T3417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 592 T3438 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:54841. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 593 T3438 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:54841, initiating session
[junit4:junit4]   2> 593 T3419 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:54846
[junit4:junit4]   2> 593 T3419 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:54846
[junit4:junit4]   2> 594 T3421 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c8f9d49f40003 with negotiated timeout 20000 for client /140.211.11.196:54846
[junit4:junit4]   2> 595 T3438 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:54841, sessionid = 0x13c8f9d49f40003, negotiated timeout = 20000
[junit4:junit4]   2> 595 T3439 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5d4a5bd9 name:ZooKeeperConnection Watcher:127.0.0.1:54841/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 595 T3417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 597 T3422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c8f9d49f40003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 606 T3417 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 608 T3417 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54844_d_tj
[junit4:junit4]   2> 608 T3422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c8f9d49f40003 type:delete cxid:0x7 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:54844_d_tj Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:54844_d_tj
[junit4:junit4]   2> 610 T3417 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:54844_d_tj
[junit4:junit4]   2> 619 T3417 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 631 T3422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c8f9d49f40003 type:delete cxid:0x15 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 632 T3417 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 635 T3417 oasc.Overseer.start Overseer (id=89103998353932291-127.0.0.1:54844_d_tj-n_0000000000) starting
[junit4:junit4]   2> 636 T3422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c8f9d49f40003 type:create cxid:0x1a zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 642 T3422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c8f9d49f40003 type:create cxid:0x1b zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 643 T3422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c8f9d49f40003 type:create cxid:0x1c zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 650 T3422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c8f9d49f40003 type:create cxid:0x1d zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 651 T3441 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 652 T3417 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 663 T3417 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 669 T3442 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1359619115791/collection1
[junit4:junit4]   2> 670 T3440 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 670 T3442 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 670 T3442 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 671 T3442 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 672 T3442 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1359619115791/collection1/'
[junit4:junit4]   2> 673 T3442 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-controljetty-1359619115791/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 674 T3442 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-controljetty-1359619115791/collection1/lib/README' to classloader
[junit4:junit4]   2> 729 T3442 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 801 T3442 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 902 T3442 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 909 T3442 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 1584 T3442 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1596 T3442 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1600 T3442 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1612 T3442 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1617 T3442 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1621 T3442 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1622 T3442 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1623 T3442 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1359619115791/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1359619115501/control/data/
[junit4:junit4]   2> 1623 T3442 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@25ae3e2d
[junit4:junit4]   2> 1624 T3442 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 1625 T3442 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/control/data forceNew:false
[junit4:junit4]   2> 1626 T3442 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/control/data
[junit4:junit4]   2> 1626 T3442 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1359619115501/control/data/index/
[junit4:junit4]   2> 1627 T3442 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1359619115501/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1628 T3442 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/control/data/index forceNew:false
[junit4:junit4]   2> 1634 T3442 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359619115501/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@76b806fe; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 1635 T3442 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 1636 T3442 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/control/data/index
[junit4:junit4]   2> 1636 T3442 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 1638 T3442 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1639 T3442 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 1639 T3442 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1639 T3442 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 1640 T3442 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1640 T3442 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1641 T3442 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1641 T3442 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1642 T3442 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1642 T3442 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 1643 T3442 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 1643 T3442 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1644 T3442 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 1644 T3442 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 1644 T3442 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 1645 T3442 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1645 T3442 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1646 T3442 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1646 T3442 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1647 T3442 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1647 T3442 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1648 T3442 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1648 T3442 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1649 T3442 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1649 T3442 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 1650 T3442 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 1663 T3442 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1667 T3442 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 1668 T3442 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 1671 T3442 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/control/data
[junit4:junit4]   2> 1673 T3442 oass.SolrIndexSearcher.<init> Opening Searcher@30d1ea07 main
[junit4:junit4]   2> 1674 T3442 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1674 T3442 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1675 T3442 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 1692 T3442 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 1734 T3443 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@30d1ea07 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1739 T3442 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 1739 T3442 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 1740 T3422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c8f9d49f40003 type:create cxid:0x50 zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 2173 T3440 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2174 T3440 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:54844_d_tj",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:54844/d_tj"}
[junit4:junit4]   2> 2174 T3440 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4]   2> 2174 T3440 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 2175 T3422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c8f9d49f40003 type:create cxid:0x58 zxid:0x2c txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 2194 T3439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 2753 T3442 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 2753 T3442 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:54844/d_tj collection:control_collection shard:shard1
[junit4:junit4]   2> 2754 T3442 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4]   2> 2772 T3422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c8f9d49f40003 type:delete cxid:0x72 zxid:0x36 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4]   2> 2773 T3442 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 2773 T3422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c8f9d49f40003 type:create cxid:0x73 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2776 T3442 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 2776 T3442 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 2776 T3442 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:54844/d_tj/collection1/
[junit4:junit4]   2> 2777 T3442 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 2777 T3442 oasc.SyncStrategy.syncToMe http://127.0.0.1:54844/d_tj/collection1/ has no replicas
[junit4:junit4]   2> 2777 T3442 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:54844/d_tj/collection1/
[junit4:junit4]   2> 2778 T3442 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4]   2> 2781 T3422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c8f9d49f40003 type:create cxid:0x7d zxid:0x3b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3697 T3440 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3715 T3439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 3752 T3442 oasc.ZkController.register We are http://127.0.0.1:54844/d_tj/collection1/ and leader is http://127.0.0.1:54844/d_tj/collection1/
[junit4:junit4]   2> 3752 T3442 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:54844/d_tj
[junit4:junit4]   2> 3752 T3442 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 3752 T3442 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 3753 T3442 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 3755 T3442 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3756 T3417 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 3757 T3417 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 3757 T3417 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 3771 T3417 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 3780 T3417 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54841/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@4dfe191e
[junit4:junit4]   2> 3781 T3417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 3782 T3444 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:54841. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 3782 T3444 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:54841, initiating session
[junit4:junit4]   2> 3782 T3419 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:54847
[junit4:junit4]   2> 3783 T3419 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:54847
[junit4:junit4]   2> 3784 T3421 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c8f9d49f40004 with negotiated timeout 10000 for client /140.211.11.196:54847
[junit4:junit4]   2> 3784 T3444 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:54841, sessionid = 0x13c8f9d49f40004, negotiated timeout = 10000
[junit4:junit4]   2> 3784 T3445 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4dfe191e name:ZooKeeperConnection Watcher:127.0.0.1:54841/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 3785 T3417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 3785 T3417 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 3787 T3417 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 3959 T3417 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 3962 T3417 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:54848
[junit4:junit4]   2> 3963 T3417 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 3964 T3417 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 3964 T3417 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1359619119283
[junit4:junit4]   2> 3965 T3417 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1359619119283/solr.xml
[junit4:junit4]   2> 3965 T3417 oasc.CoreContainer.<init> New CoreContainer 318437194
[junit4:junit4]   2> 3966 T3417 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1359619119283/'
[junit4:junit4]   2> 3967 T3417 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1359619119283/'
[junit4:junit4]   2> 4024 T3417 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 4025 T3417 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 4026 T3417 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 4026 T3417 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 4027 T3417 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 4027 T3417 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 4028 T3417 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 4028 T3417 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 4029 T3417 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 4029 T3417 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 4050 T3417 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 4070 T3417 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:54841/solr
[junit4:junit4]   2> 4071 T3417 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 4071 T3417 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54841 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@68719da6
[junit4:junit4]   2> 4073 T3417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4073 T3455 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:54841. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4074 T3455 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:54841, initiating session
[junit4:junit4]   2> 4074 T3419 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:54849
[junit4:junit4]   2> 4074 T3419 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:54849
[junit4:junit4]   2> 4090 T3421 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c8f9d49f40005 with negotiated timeout 20000 for client /140.211.11.196:54849
[junit4:junit4]   2> 4090 T3455 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:54841, sessionid = 0x13c8f9d49f40005, negotiated timeout = 20000
[junit4:junit4]   2> 4090 T3456 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@68719da6 name:ZooKeeperConnection Watcher:127.0.0.1:54841 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4090 T3417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4092 T3422 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c8f9d49f40005
[junit4:junit4]   2> 4113 T3456 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 4113 T3419 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:54849 which had sessionid 0x13c8f9d49f40005
[junit4:junit4]   2> 4113 T3417 oaz.ZooKeeper.close Session: 0x13c8f9d49f40005 closed
[junit4:junit4]   2> 4114 T3417 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 4124 T3417 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54841/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@3a9a7893
[junit4:junit4]   2> 4125 T3417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4126 T3457 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:54841. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4126 T3457 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:54841, initiating session
[junit4:junit4]   2> 4127 T3419 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:54850
[junit4:junit4]   2> 4127 T3419 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:54850
[junit4:junit4]   2> 4128 T3421 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c8f9d49f40006 with negotiated timeout 20000 for client /140.211.11.196:54850
[junit4:junit4]   2> 4128 T3457 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:54841, sessionid = 0x13c8f9d49f40006, negotiated timeout = 20000
[junit4:junit4]   2> 4129 T3458 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3a9a7893 name:ZooKeeperConnection Watcher:127.0.0.1:54841/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4129 T3417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4130 T3422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c8f9d49f40006 type:create cxid:0x1 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4161 T3422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c8f9d49f40006 type:create cxid:0x2 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4162 T3417 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54848_d_tj
[junit4:junit4]   2> 4163 T3422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c8f9d49f40006 type:delete cxid:0x4 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:54848_d_tj Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:54848_d_tj
[junit4:junit4]   2> 4203 T3417 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:54848_d_tj
[junit4:junit4]   2> 4206 T3439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 4206 T3445 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 4207 T3439 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 4209 T3417 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 4216 T3459 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1359619119283/collection1
[junit4:junit4]   2> 4216 T3459 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 4217 T3459 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 4217 T3459 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 4218 T3459 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1359619119283/collection1/'
[junit4:junit4]   2> 4220 T3459 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty1-1359619119283/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 4220 T3459 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty1-1359619119283/collection1/lib/README' to classloader
[junit4:junit4]   2> 4275 T3459 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 4348 T3459 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 4449 T3459 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 4455 T3459 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 5103 T3459 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 5115 T3459 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 5119 T3459 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 5131 T3459 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5136 T3459 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5140 T3459 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5141 T3459 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5142 T3459 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1359619119283/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty1/
[junit4:junit4]   2> 5142 T3459 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@25ae3e2d
[junit4:junit4]   2> 5143 T3459 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 5144 T3459 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty1 forceNew:false
[junit4:junit4]   2> 5145 T3459 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty1
[junit4:junit4]   2> 5145 T3459 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty1/index/
[junit4:junit4]   2> 5145 T3459 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 5146 T3459 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty1/index forceNew:false
[junit4:junit4]   2> 5152 T3459 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@549256d3; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 5153 T3459 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 5154 T3459 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty1/index
[junit4:junit4]   2> 5154 T3459 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 5157 T3459 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 5157 T3459 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 5157 T3459 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 5158 T3459 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 5158 T3459 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 5159 T3459 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 5159 T3459 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 5160 T3459 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 5160 T3459 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 5161 T3459 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 5162 T3459 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 5162 T3459 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 5163 T3459 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 5163 T3459 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 5163 T3459 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 5164 T3459 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 5165 T3459 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5165 T3459 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5166 T3459 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5166 T3459 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5167 T3459 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5167 T3459 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5168 T3459 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5169 T3459 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5169 T3459 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 5170 T3459 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 5183 T3459 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 5188 T3459 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 5189 T3459 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 5191 T3459 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty1
[junit4:junit4]   2> 5193 T3459 oass.SolrIndexSearcher.<init> Opening Searcher@10af63 main
[junit4:junit4]   2> 5194 T3459 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 5195 T3459 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 5195 T3459 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 5203 T3459 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 5220 T3440 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 5221 T3440 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:54844_d_tj",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:54844/d_tj"}
[junit4:junit4]   2> 5238 T3458 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5238 T3445 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5238 T3439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5271 T3460 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@10af63 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 5277 T3459 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 5277 T3459 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 6747 T3440 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6748 T3440 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:54848_d_tj",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:54848/d_tj"}
[junit4:junit4]   2> 6748 T3440 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 1
[junit4:junit4]   2> 6748 T3440 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 6766 T3445 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6766 T3458 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6766 T3439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7281 T3459 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 7281 T3459 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:54848/d_tj collection:collection1 shard:shard1
[junit4:junit4]   2> 7282 T3459 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 7299 T3422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c8f9d49f40006 type:delete cxid:0x45 zxid:0x5b txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 7300 T3459 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 7301 T3422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c8f9d49f40006 type:create cxid:0x46 zxid:0x5c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7302 T3459 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 7303 T3459 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 7303 T3459 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:54848/d_tj/collection1/
[junit4:junit4]   2> 7303 T3459 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 7304 T3459 oasc.SyncStrategy.syncToMe http://127.0.0.1:54848/d_tj/collection1/ has no replicas
[junit4:junit4]   2> 7304 T3459 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:54848/d_tj/collection1/
[junit4:junit4]   2> 7304 T3459 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 7307 T3422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c8f9d49f40006 type:create cxid:0x50 zxid:0x60 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8270 T3440 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8289 T3439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8289 T3445 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8289 T3458 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8295 T3459 oasc.ZkController.register We are http://127.0.0.1:54848/d_tj/collection1/ and leader is http://127.0.0.1:54848/d_tj/collection1/
[junit4:junit4]   2> 8295 T3459 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:54848/d_tj
[junit4:junit4]   2> 8295 T3459 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 8295 T3459 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 8296 T3459 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 8298 T3459 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8299 T3417 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 8300 T3417 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 8300 T3417 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 8479 T3417 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 8483 T3417 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:54852
[junit4:junit4]   2> 8484 T3417 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 8484 T3417 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 8485 T3417 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1359619123810
[junit4:junit4]   2> 8485 T3417 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1359619123810/solr.xml
[junit4:junit4]   2> 8486 T3417 oasc.CoreContainer.<init> New CoreContainer 586793291
[junit4:junit4]   2> 8487 T3417 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1359619123810/'
[junit4:junit4]   2> 8487 T3417 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1359619123810/'
[junit4:junit4]   2> 8544 T3417 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 8545 T3417 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 8545 T3417 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 8546 T3417 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 8546 T3417 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 8547 T3417 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 8547 T3417 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 8548 T3417 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 8548 T3417 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 8549 T3417 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 8575 T3417 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 8595 T3417 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:54841/solr
[junit4:junit4]   2> 8596 T3417 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 8596 T3417 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54841 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@2c2ce77
[junit4:junit4]   2> 8597 T3417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 8598 T3470 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:54841. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 8598 T3470 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:54841, initiating session
[junit4:junit4]   2> 8599 T3419 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:54853
[junit4:junit4]   2> 8599 T3419 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:54853
[junit4:junit4]   2> 8601 T3421 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c8f9d49f40007 with negotiated timeout 20000 for client /140.211.11.196:54853
[junit4:junit4]   2> 8601 T3470 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:54841, sessionid = 0x13c8f9d49f40007, negotiated timeout = 20000
[junit4:junit4]   2> 8601 T3471 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2c2ce77 name:ZooKeeperConnection Watcher:127.0.0.1:54841 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 8602 T3417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 8603 T3422 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c8f9d49f40007
[junit4:junit4]   2> 8612 T3471 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 8612 T3419 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:54853 which had sessionid 0x13c8f9d49f40007
[junit4:junit4]   2> 8612 T3417 oaz.ZooKeeper.close Session: 0x13c8f9d49f40007 closed
[junit4:junit4]   2> 8613 T3417 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 8624 T3417 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54841/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@77a29b6d
[junit4:junit4]   2> 8625 T3417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 8626 T3472 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:54841. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 8627 T3472 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:54841, initiating session
[junit4:junit4]   2> 8627 T3419 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:54854
[junit4:junit4]   2> 8627 T3419 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:54854
[junit4:junit4]   2> 8628 T3421 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c8f9d49f40008 with negotiated timeout 20000 for client /140.211.11.196:54854
[junit4:junit4]   2> 8628 T3472 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:54841, sessionid = 0x13c8f9d49f40008, negotiated timeout = 20000
[junit4:junit4]   2> 8629 T3473 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@77a29b6d name:ZooKeeperConnection Watcher:127.0.0.1:54841/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 8629 T3417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 8630 T3422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c8f9d49f40008 type:create cxid:0x1 zxid:0x6d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8636 T3422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c8f9d49f40008 type:create cxid:0x2 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8638 T3417 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54852_d_tj
[junit4:junit4]   2> 8639 T3422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c8f9d49f40008 type:delete cxid:0x4 zxid:0x6f txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:54852_d_tj Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:54852_d_tj
[junit4:junit4]   2> 8640 T3417 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:54852_d_tj
[junit4:junit4]   2> 8648 T3439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8649 T3458 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 8649 T3445 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 8649 T3458 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8649 T3445 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8651 T3439 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 8651 T3417 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 8658 T3474 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1359619123810/collection1
[junit4:junit4]   2> 8659 T3474 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 8659 T3474 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 8660 T3474 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 8661 T3474 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1359619123810/collection1/'
[junit4:junit4]   2> 8662 T3474 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty2-1359619123810/collection1/lib/README' to classloader
[junit4:junit4]   2> 8663 T3474 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty2-1359619123810/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 8718 T3474 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 8790 T3474 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 8891 T3474 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 8898 T3474 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 9552 T3474 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 9564 T3474 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 9568 T3474 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 9580 T3474 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 9586 T3474 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 9590 T3474 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 9591 T3474 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 9592 T3474 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1359619123810/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty2/
[junit4:junit4]   2> 9593 T3474 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@25ae3e2d
[junit4:junit4]   2> 9593 T3474 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 9594 T3474 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty2 forceNew:false
[junit4:junit4]   2> 9595 T3474 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty2
[junit4:junit4]   2> 9595 T3474 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty2/index/
[junit4:junit4]   2> 9596 T3474 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 9597 T3474 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty2/index forceNew:false
[junit4:junit4]   2> 9610 T3474 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@41286b1b; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 9610 T3474 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 9611 T3474 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty2/index
[junit4:junit4]   2> 9612 T3474 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 9614 T3474 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 9614 T3474 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 9615 T3474 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 9615 T3474 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 9616 T3474 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 9616 T3474 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 9617 T3474 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 9617 T3474 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 9618 T3474 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 9618 T3474 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 9619 T3474 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 9620 T3474 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 9620 T3474 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 9620 T3474 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 9621 T3474 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 9621 T3474 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 9622 T3474 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9623 T3474 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9623 T3474 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9624 T3474 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9624 T3474 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9625 T3474 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9626 T3474 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9626 T3474 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9627 T3474 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 9627 T3474 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 9640 T3474 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 9645 T3474 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 9646 T3474 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 9649 T3474 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty2
[junit4:junit4]   2> 9651 T3474 oass.SolrIndexSearcher.<init> Opening Searcher@34a26293 main
[junit4:junit4]   2> 9652 T3474 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 9652 T3474 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 9653 T3474 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 9660 T3474 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 9717 T3475 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@34a26293 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 9723 T3474 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 9723 T3474 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 9794 T3440 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9795 T3440 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:54848_d_tj",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:54848/d_tj"}
[junit4:junit4]   2> 9798 T3440 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:54852_d_tj",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:54852/d_tj"}
[junit4:junit4]   2> 9798 T3440 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4]   2> 9799 T3440 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 9810 T3473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9810 T3439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9810 T3458 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9810 T3445 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10727 T3474 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 10727 T3474 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:54852/d_tj collection:collection1 shard:shard1
[junit4:junit4]   2> 10730 T3474 oasc.ZkController.register We are http://127.0.0.1:54852/d_tj/collection1/ and leader is http://127.0.0.1:54848/d_tj/collection1/
[junit4:junit4]   2> 10731 T3474 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:54852/d_tj
[junit4:junit4]   2> 10731 T3474 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 10731 T3474 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C210 name=collection1 org.apache.solr.core.SolrCore@45409b13 url=http://127.0.0.1:54852/d_tj/collection1 node=127.0.0.1:54852_d_tj C210_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:54852_d_tj, base_url=http://127.0.0.1:54852/d_tj}
[junit4:junit4]   2> 10731 T3476 C210 P54852 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 10732 T3474 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 10733 T3476 C210 P54852 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 10733 T3476 C210 P54852 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 10733 T3476 C210 P54852 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 10734 T3417 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 10734 T3417 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 10735 T3476 C210 P54852 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 10735 T3417 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 10921 T3417 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 10924 T3417 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:54857
[junit4:junit4]   2> 10925 T3417 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 10925 T3417 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 10926 T3417 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1359619126251
[junit4:junit4]   2> 10926 T3417 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1359619126251/solr.xml
[junit4:junit4]   2> 10927 T3417 oasc.CoreContainer.<init> New CoreContainer 537761026
[junit4:junit4]   2> 10928 T3417 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1359619126251/'
[junit4:junit4]   2> 10928 T3417 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1359619126251/'
[junit4:junit4]   2> 10985 T3417 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 10986 T3417 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 10986 T3417 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 10987 T3417 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 10987 T3417 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 10988 T3417 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 10988 T3417 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 10989 T3417 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 10989 T3417 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 10990 T3417 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 11010 T3417 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 11029 T3417 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:54841/solr
[junit4:junit4]   2> 11030 T3417 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 11031 T3417 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54841 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@41f3922a
[junit4:junit4]   2> 11032 T3417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 11032 T3486 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:54841. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 11033 T3486 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:54841, initiating session
[junit4:junit4]   2> 11033 T3419 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:54858
[junit4:junit4]   2> 11033 T3419 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:54858
[junit4:junit4]   2> 11035 T3421 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c8f9d49f40009 with negotiated timeout 20000 for client /140.211.11.196:54858
[junit4:junit4]   2> 11035 T3486 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:54841, sessionid = 0x13c8f9d49f40009, negotiated timeout = 20000
[junit4:junit4]   2> 11036 T3487 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@41f3922a name:ZooKeeperConnection Watcher:127.0.0.1:54841 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 11036 T3417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 11037 T3422 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c8f9d49f40009
[junit4:junit4]   2> 11047 T3487 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 11048 T3419 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:54858 which had sessionid 0x13c8f9d49f40009
[junit4:junit4]   2> 11047 T3417 oaz.ZooKeeper.close Session: 0x13c8f9d49f40009 closed
[junit4:junit4]   2> 11048 T3417 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 11058 T3417 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54841/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@b1892d0
[junit4:junit4]   2> 11059 T3417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 11059 T3488 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:54841. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 11060 T3488 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:54841, initiating session
[junit4:junit4]   2> 11060 T3419 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:54859
[junit4:junit4]   2> 11060 T3419 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:54859
[junit4:junit4]   2> 11061 T3421 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c8f9d49f4000a with negotiated timeout 20000 for client /140.211.11.196:54859
[junit4:junit4]   2> 11061 T3488 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:54841, sessionid = 0x13c8f9d49f4000a, negotiated timeout = 20000
[junit4:junit4]   2> 11062 T3489 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b1892d0 name:ZooKeeperConnection Watcher:127.0.0.1:54841/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 11062 T3417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 11063 T3422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c8f9d49f4000a type:create cxid:0x1 zxid:0x7f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 11064 T3422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c8f9d49f4000a type:create cxid:0x2 zxid:0x80 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 11066 T3417 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54857_d_tj
[junit4:junit4]   2> 11067 T3422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c8f9d49f4000a type:delete cxid:0x4 zxid:0x81 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:54857_d_tj Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:54857_d_tj
[junit4:junit4]   2> 11075 T3417 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:54857_d_tj
[junit4:junit4]   2> 11078 T3439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11078 T3458 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 11078 T3473 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 11078 T3458 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11078 T3445 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 11078 T3473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11079 T3445 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11079 T3439 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 11081 T3417 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 11088 T3490 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1359619126251/collection1
[junit4:junit4]   2> 11088 T3490 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 11089 T3490 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 11089 T3490 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 11090 T3490 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1359619126251/collection1/'
[junit4:junit4]   2> 11092 T3490 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty3-1359619126251/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 11092 T3490 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty3-1359619126251/collection1/lib/README' to classloader
[junit4:junit4]   2> 11147 T3490 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 11220 T3490 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 11316 T3440 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11317 T3440 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:54852_d_tj",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:54852/d_tj"}
[junit4:junit4]   2> 11321 T3490 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 11327 T3490 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 11333 T3489 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11333 T3445 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11333 T3473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11333 T3458 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11333 T3439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11967 T3490 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 11978 T3490 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 11982 T3490 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 11995 T3490 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 12000 T3490 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 12004 T3490 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 12005 T3490 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 12006 T3490 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1359619126251/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty3/
[junit4:junit4]   2> 12007 T3490 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@25ae3e2d
[junit4:junit4]   2> 12007 T3490 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 12008 T3490 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty3 forceNew:false
[junit4:junit4]   2> 12009 T3490 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty3
[junit4:junit4]   2> 12009 T3490 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty3/index/
[junit4:junit4]   2> 12010 T3490 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 12011 T3490 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty3/index forceNew:false
[junit4:junit4]   2> 12017 T3490 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@52265fb8; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 12017 T3490 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 12018 T3490 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty3/index
[junit4:junit4]   2> 12019 T3490 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 12021 T3490 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 12021 T3490 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 12022 T3490 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 12022 T3490 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 12023 T3490 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 12023 T3490 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 12023 T3490 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 12024 T3490 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 12025 T3490 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 12025 T3490 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 12026 T3490 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 12026 T3490 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 12027 T3490 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 12027 T3490 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 12028 T3490 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 12028 T3490 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 12029 T3490 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12029 T3490 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12030 T3490 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12031 T3490 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12031 T3490 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12032 T3490 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12032 T3490 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12033 T3490 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12034 T3490 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 12034 T3490 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 12047 T3490 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 12052 T3490 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 12053 T3490 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 12055 T3490 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty3
[junit4:junit4]   2> 12057 T3490 oass.SolrIndexSearcher.<init> Opening Searcher@446bc840 main
[junit4:junit4]   2> 12058 T3490 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 12058 T3490 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 12059 T3490 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 12066 T3490 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 12116 T3491 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@446bc840 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 12122 T3490 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 12122 T3490 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 12837 T3440 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12838 T3440 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:54857_d_tj",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:54857/d_tj"}
[junit4:junit4]   2> 12838 T3440 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4]   2> 12839 T3440 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 12855 T3439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12855 T3458 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12855 T3473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12855 T3489 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12855 T3445 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13125 T3490 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 13125 T3490 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:54857/d_tj collection:collection1 shard:shard1
[junit4:junit4]   2> 13128 T3490 oasc.ZkController.register We are http://127.0.0.1:54857/d_tj/collection1/ and leader is http://127.0.0.1:54848/d_tj/collection1/
[junit4:junit4]   2> 13128 T3490 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:54857/d_tj
[junit4:junit4]   2> 13128 T3490 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 13129 T3490 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C211 name=collection1 org.apache.solr.core.SolrCore@16a6270c url=http://127.0.0.1:54857/d_tj/collection1 node=127.0.0.1:54857_d_tj C211_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:54857_d_tj, base_url=http://127.0.0.1:54857/d_tj}
[junit4:junit4]   2> 13129 T3492 C211 P54857 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 13129 T3490 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 13131 T3492 C211 P54857 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 13131 T3492 C211 P54857 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 13131 T3492 C211 P54857 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 13131 T3417 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 13132 T3417 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 13133 T3417 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 13134 T3492 C211 P54857 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 13314 T3417 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 13317 T3417 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:54861
[junit4:junit4]   2> 13318 T3417 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 13318 T3417 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 13319 T3417 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1359619128645
[junit4:junit4]   2> 13319 T3417 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1359619128645/solr.xml
[junit4:junit4]   2> 13320 T3417 oasc.CoreContainer.<init> New CoreContainer 226747186
[junit4:junit4]   2> 13320 T3417 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1359619128645/'
[junit4:junit4]   2> 13321 T3417 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1359619128645/'
[junit4:junit4]   2> 13378 T3417 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 13379 T3417 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 13379 T3417 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 13380 T3417 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 13380 T3417 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 13381 T3417 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 13381 T3417 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 13382 T3417 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 13382 T3417 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 13383 T3417 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 13403 T3417 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 13423 T3417 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:54841/solr
[junit4:junit4]   2> 13423 T3417 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 13424 T3417 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54841 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@7c46ad8
[junit4:junit4]   2> 13425 T3417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 13425 T3502 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:54841. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 13426 T3502 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:54841, initiating session
[junit4:junit4]   2> 13426 T3419 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:54862
[junit4:junit4]   2> 13427 T3419 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:54862
[junit4:junit4]   2> 13429 T3421 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c8f9d49f4000b with negotiated timeout 20000 for client /140.211.11.196:54862
[junit4:junit4]   2> 13429 T3502 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:54841, sessionid = 0x13c8f9d49f4000b, negotiated timeout = 20000
[junit4:junit4]   2> 13429 T3503 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7c46ad8 name:ZooKeeperConnection Watcher:127.0.0.1:54841 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 13430 T3417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 13431 T3422 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c8f9d49f4000b
[junit4:junit4]   2> 13444 T3503 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 13444 T3419 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:54862 which had sessionid 0x13c8f9d49f4000b
[junit4:junit4]   2> 13444 T3417 oaz.ZooKeeper.close Session: 0x13c8f9d49f4000b closed
[junit4:junit4]   2> 13444 T3417 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 13454 T3417 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:54841/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@49621238
[junit4:junit4]   2> 13455 T3417 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 13456 T3504 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:54841. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 13456 T3504 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:54841, initiating session
[junit4:junit4]   2> 13457 T3419 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:54863
[junit4:junit4]   2> 13457 T3419 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:54863
[junit4:junit4]   2> 13458 T3421 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c8f9d49f4000c with negotiated timeout 20000 for client /140.211.11.196:54863
[junit4:junit4]   2> 13458 T3504 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:54841, sessionid = 0x13c8f9d49f4000c, negotiated timeout = 20000
[junit4:junit4]   2> 13459 T3505 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@49621238 name:ZooKeeperConnection Watcher:127.0.0.1:54841/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 13459 T3417 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 13460 T3422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c8f9d49f4000c type:create cxid:0x1 zxid:0x92 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 13461 T3422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c8f9d49f4000c type:create cxid:0x2 zxid:0x93 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 13463 T3417 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54861_d_tj
[junit4:junit4]   2> 13463 T3422 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c8f9d49f4000c type:delete cxid:0x4 zxid:0x94 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:54861_d_tj Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:54861_d_tj
[junit4:junit4]   2> 13471 T3417 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:54861_d_tj
[junit4:junit4]   2> 13473 T3439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13473 T3489 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13473 T3458 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 13473 T3473 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 13473 T3445 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 13474 T3458 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 13474 T3445 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 13474 T3473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 13475 T3439 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 13476 T3489 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 13477 T3417 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 13484 T3506 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1359619128645/collection1
[junit4:junit4]   2> 13484 T3506 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 13485 T3506 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 13485 T3506 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 13486 T3506 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1359619128645/collection1/'
[junit4:junit4]   2> 13487 T3506 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty4-1359619128645/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 13488 T3506 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty4-1359619128645/collection1/lib/README' to classloader
[junit4:junit4]   2> 13542 T3506 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 13615 T3506 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 13716 T3506 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 13724 T3506 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2>  C210_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:54852_d_tj, base_url=http://127.0.0.1:54852/d_tj}
[junit4:junit4]   2> 13759 T3476 C210 P54852 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:54848/d_tj/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 13759 T3476 C210 P54852 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:54852/d_tj START replicas=[http://127.0.0.1:54848/d_tj/collection1/] nUpdates=100
[junit4:junit4]   2> 13760 T3476 C210 P54852 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 13760 T3476 C210 P54852 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 13760 T3476 C210 P54852 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 13760 T3476 C210 P54852 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 13761 T3476 C210 P54852 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 13761 T3476 C210 P54852 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:54848/d_tj/collection1/. core=collection1
[junit4:junit4]   2> 13761 T3476 C210 P54852 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C212 name=collection1 org.apache.solr.core.SolrCore@16aea2c6 url=http://127.0.0.1:54848/d_tj/collection1 node=127.0.0.1:54848_d_tj C212_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:54848_d_tj, base_url=http://127.0.0.1:54848/d_tj, leader=true}
[junit4:junit4]   2> 13770 T3450 C212 P54848 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=2 
[junit4:junit4]   2> 13773 T3451 C212 P54848 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 13775 T3451 C212 P54848 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty1
[junit4:junit4]   2> 13780 T3451 C212 P54848 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@549256d3; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 13781 T3451 C212 P54848 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 13786 T3451 C212 P54848 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@549256d3; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@549256d3; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 13787 T3451 C212 P54848 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 13788 T3451 C212 P54848 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty1
[junit4:junit4]   2> 13789 T3451 C212 P54848 oass.SolrIndexSearcher.<init> Opening Searcher@11cb9ba2 realtime
[junit4:junit4]   2> 13789 T3451 C212 P54848 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 13789 T3451 C212 P54848 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 16
[junit4:junit4]   2> 13790 T3476 C210 P54852 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 13791 T3476 C210 P54852 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 13793 T3452 C212 P54848 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 13793 T3452 C212 P54848 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 13794 T3476 C210 P54852 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 13794 T3476 C210 P54852 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 13794 T3476 C210 P54852 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 13797 T3452 C212 P54848 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty1
[junit4:junit4]   2> 13798 T3452 C212 P54848 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty1/index
[junit4:junit4]   2> 13799 T3452 C212 P54848 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=2 
[junit4:junit4]   2> 13800 T3476 C210 P54852 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 13801 T3476 C210 P54852 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty2/index.20130130225849296 forceNew:false
[junit4:junit4]   2> 13802 T3476 C210 P54852 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty2
[junit4:junit4]   2> 13803 T3476 C210 P54852 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty2/index.20130130225849296 lockFactory=org.apache.lucene.store.NativeFSLockFactory@10418723; maxCacheMB=48.0 maxMergeSizeMB=4.0)) fullCopy=true
[junit4:junit4]   2> 13806 T3452 C212 P54848 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> 13808 T3476 C210 P54852 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty2/index
[junit4:junit4]   2> 13809 T3476 C210 P54852 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 13815 T3476 C210 P54852 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130130225849296
[junit4:junit4]   2> 13817 T3476 C210 P54852 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty2
[junit4:junit4]   2> 13817 T3476 C210 P54852 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty2
[junit4:junit4]   2> 13818 T3476 C210 P54852 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty2/index/ new=./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty2/index.20130130225849296
[junit4:junit4]   2> 13821 T3476 C210 P54852 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty2/index.20130130225849296 lockFactory=org.apache.lucene.store.NativeFSLockFactory@10418723; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 13822 T3476 C210 P54852 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 13823 T3476 C210 P54852 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty2
[junit4:junit4]   2> 13824 T3476 C210 P54852 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 13824 T3476 C210 P54852 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 13824 T3476 C210 P54852 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 13825 T3476 C210 P54852 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty2/index.20130130225849296
[junit4:junit4]   2> 13826 T3476 C210 P54852 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty2
[junit4:junit4]   2> 13831 T3476 C210 P54852 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty2/index.20130130225849296 lockFactory=org.apache.lucene.store.NativeFSLockFactory@10418723; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 13832 T3476 C210 P54852 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 13832 T3476 C210 P54852 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 13833 T3476 C210 P54852 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty2
[junit4:junit4]   2> 13833 T3476 C210 P54852 oass.SolrIndexSearcher.<init> Opening Searcher@2a695c3 main
[junit4:junit4]   2> 13835 T3475 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2a695c3 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 13836 T3475 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty2/index
[junit4:junit4]   2> 13836 T3476 C210 P54852 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 13900 T3476 C210 P54852 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty2/index.20130130225849296 lockFactory=org.apache.lucene.store.NativeFSLockFactory@10418723; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty2/index.20130130225849296 lockFactory=org.apache.lucene.store.NativeFSLockFactory@10418723; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 13901 T3476 C210 P54852 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 13902 T3476 C210 P54852 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty2
[junit4:junit4]   2> 13903 T3476 C210 P54852 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 13903 T3476 C210 P54852 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty2/index.20130130225849296
[junit4:junit4]   2> 13903 T3476 C210 P54852 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty2/index
[junit4:junit4]   2> 13903 T3476 C210 P54852 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 13904 T3476 C210 P54852 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 13904 T3476 C210 P54852 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 13904 T3476 C210 P54852 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 13912 T3476 C210 P54852 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 14359 T3440 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14360 T3440 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:54857_d_tj",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:54857/d_tj"}
[junit4:junit4]   2> 14367 T3440 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:54852_d_tj",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:54852/d_tj"}
[junit4:junit4]   2> 14376 T3506 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 14379 T3439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14379 T3489 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14379 T3505 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14379 T3473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14379 T3458 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14379 T3445 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14387 T3506 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 14391 T3506 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 14402 T3506 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 14417 T3506 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 14421 T3506 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 14422 T3506 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 14423 T3506 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1359619128645/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty4/
[junit4:junit4]   2> 14423 T3506 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@25ae3e2d
[junit4:junit4]   2> 14424 T3506 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 14425 T3506 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty4 forceNew:false
[junit4:junit4]   2> 14425 T3506 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty4
[junit4:junit4]   2> 14426 T3506 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty4/index/
[junit4:junit4]   2> 14426 T3506 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 14427 T3506 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty4/index forceNew:false
[junit4:junit4]   2> 14435 T3506 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2e9af0e7; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 14436 T3506 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 14437 T3506 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty4/index
[junit4:junit4]   2> 14437 T3506 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 14439 T3506 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 14440 T3506 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 14440 T3506 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 14440 T3506 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 14441 T3506 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 14441 T3506 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 14442 T3506 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 14442 T3506 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 14443 T3506 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 14443 T3506 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 14444 T3506 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 14444 T3506 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 14445 T3506 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 14445 T3506 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 14446 T3506 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 14446 T3506 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 14447 T3506 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14447 T3506 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14448 T3506 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14448 T3506 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14449 T3506 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14449 T3506 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14450 T3506 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14451 T3506 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14451 T3506 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 14452 T3506 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 14464 T3506 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 14468 T3506 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 14469 T3506 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 14472 T3506 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty4
[junit4:junit4]   2> 14473 T3506 oass.SolrIndexSearcher.<init> Opening Searcher@613351d6 main
[junit4:junit4]   2> 14474 T3506 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 14475 T3506 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 14475 T3506 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 14486 T3506 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 14563 T3509 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@613351d6 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 14568 T3506 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 14568 T3506 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 15892 T3440 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 15893 T3440 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:54861_d_tj",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:54861/d_tj"}
[junit4:junit4]   2> 15893 T3440 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4]   2> 15894 T3440 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 15912 T3439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15912 T3473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15912 T3458 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15912 T3505 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15912 T3445 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15912 T3489 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16572 T3506 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 16572 T3506 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:54861/d_tj collection:collection1 shard:shard1
[junit4:junit4]   2> 16575 T3506 oasc.ZkController.register We are http://127.0.0.1:54861/d_tj/collection1/ and leader is http://127.0.0.1:54848/d_tj/collection1/
[junit4:junit4]   2> 16575 T3506 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:54861/d_tj
[junit4:junit4]   2> 16576 T3506 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 16576 T3506 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C213 name=collection1 org.apache.solr.core.SolrCore@506ff1aa url=http://127.0.0.1:54861/d_tj/collection1 node=127.0.0.1:54861_d_tj C213_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:54861_d_tj, base_url=http://127.0.0.1:54861/d_tj}
[junit4:junit4]   2> 16577 T3510 C213 P54861 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 16577 T3506 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16578 T3510 C213 P54861 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 16578 T3510 C213 P54861 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 16578 T3510 C213 P54861 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 16579 T3417 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 16579 T3417 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 16580 T3510 C213 P54861 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 16580 T3417 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 16596 T3417 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16598 T3417 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4]   2> 16598 T3417 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4]   2> 16599 T3417 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C211_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:54857_d_tj, base_url=http://127.0.0.1:54857/d_tj}
[junit4:junit4]   2> 17153 T3492 C211 P54857 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:54848/d_tj/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 17153 T3492 C211 P54857 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:54857/d_tj START replicas=[http://127.0.0.1:54848/d_tj/collection1/] nUpdates=100
[junit4:junit4]   2> 17153 T3492 C211 P54857 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 17154 T3492 C211 P54857 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 17154 T3492 C211 P54857 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 17154 T3492 C211 P54857 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 17154 T3492 C211 P54857 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 17154 T3492 C211 P54857 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:54848/d_tj/collection1/. core=collection1
[junit4:junit4]   2> 17155 T3492 C211 P54857 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 17156 T3450 C212 P54848 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 17165 T3454 C212 P54848 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 17173 T3454 C212 P54848 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@549256d3; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@549256d3; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 17173 T3454 C212 P54848 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 17174 T3454 C212 P54848 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty1
[junit4:junit4]   2> 17174 T3454 C212 P54848 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 17175 T3454 C212 P54848 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
[junit4:junit4]   2> 17176 T3492 C211 P54857 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 17176 T3492 C211 P54857 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 17178 T3452 C212 P54848 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 17179 T3492 C211 P54857 oash.SnapPuller.fetchLatestIndex Master's generation: 3
[junit4:junit4]   2> 17179 T3492 C211 P54857 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 17179 T3492 C211 P54857 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 17182 T3452 C212 P54848 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty1
[junit4:junit4]   2> 17183 T3452 C212 P54848 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty1/index
[junit4:junit4]   2> 17183 T3452 C212 P54848 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=3&version=2} status=0 QTime=1 
[junit4:junit4]   2> 17184 T3492 C211 P54857 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 17185 T3492 C211 P54857 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty3/index.20130130225852680 forceNew:false
[junit4:junit4]   2> 17186 T3492 C211 P54857 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty3
[junit4:junit4]   2> 17187 T3492 C211 P54857 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty3/index.20130130225852680 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79480033; maxCacheMB=48.0 maxMergeSizeMB=4.0)) fullCopy=true
[junit4:junit4]   2> 17190 T3452 C212 P54848 REQ /replication {file=segments_3&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=3} status=0 QTime=0 
[junit4:junit4]   2> 17191 T3492 C211 P54857 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty3/index
[junit4:junit4]   2> 17192 T3492 C211 P54857 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 17192 T3492 C211 P54857 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130130225852680
[junit4:junit4]   2> 17193 T3492 C211 P54857 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty3
[junit4:junit4]   2> 17194 T3492 C211 P54857 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty3
[junit4:junit4]   2> 17194 T3492 C211 P54857 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty3/index/ new=./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty3/index.20130130225852680
[junit4:junit4]   2> 17198 T3492 C211 P54857 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty3/index.20130130225852680 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79480033; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 17198 T3492 C211 P54857 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 17199 T3492 C211 P54857 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty3
[junit4:junit4]   2> 17200 T3492 C211 P54857 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 17200 T3492 C211 P54857 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 17200 T3492 C211 P54857 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 17201 T3492 C211 P54857 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty3/index.20130130225852680
[junit4:junit4]   2> 17202 T3492 C211 P54857 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty3
[junit4:junit4]   2> 17205 T3492 C211 P54857 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty3/index.20130130225852680 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79480033; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 17205 T3492 C211 P54857 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 17205 T3492 C211 P54857 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 17206 T3492 C211 P54857 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty3
[junit4:junit4]   2> 17206 T3492 C211 P54857 oass.SolrIndexSearcher.<init> Opening Searcher@6514ea71 main
[junit4:junit4]   2> 17208 T3491 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6514ea71 main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4]   2> 17209 T3491 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty3/index
[junit4:junit4]   2> 17209 T3492 C211 P54857 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 17212 T3492 C211 P54857 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty3/index.20130130225852680 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79480033; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty3/index.20130130225852680 lockFactory=org.apache.lucene.store.NativeFSLockFactory@79480033; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_4,generation=4,filenames=[segments_4]
[junit4:junit4]   2> 17213 T3492 C211 P54857 oasc.SolrDeletionPolicy.updateCommits newest commit = 4[segments_4]
[junit4:junit4]   2> 17214 T3492 C211 P54857 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty3
[junit4:junit4]   2> 17214 T3492 C211 P54857 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 17214 T3492 C211 P54857 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty3/index.20130130225852680
[junit4:junit4]   2> 17214 T3492 C211 P54857 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty3/index
[junit4:junit4]   2> 17215 T3492 C211 P54857 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 17215 T3492 C211 P54857 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 17215 T3492 C211 P54857 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 17215 T3492 C211 P54857 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 17222 T3492 C211 P54857 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 17416 T3440 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17417 T3440 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:54861_d_tj",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:54861/d_tj"}
[junit4:junit4]   2> 17420 T3440 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:54857_d_tj",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:54857/d_tj"}
[junit4:junit4]   2> 17433 T3489 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17433 T3439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17433 T3445 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17433 T3473 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17433 T3505 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17433 T3458 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17601 T3417 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 18603 T3417 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C213_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:54861_d_tj, base_url=http://127.0.0.1:54861/d_tj}
[junit4:junit4]   2> 19599 T3510 C213 P54861 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:54848/d_tj/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 19599 T3510 C213 P54861 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:54861/d_tj START replicas=[http://127.0.0.1:54848/d_tj/collection1/] nUpdates=100
[junit4:junit4]   2> 19600 T3510 C213 P54861 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 19600 T3510 C213 P54861 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 19600 T3510 C213 P54861 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 19600 T3510 C213 P54861 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 19601 T3510 C213 P54861 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 19601 T3510 C213 P54861 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:54848/d_tj/collection1/. core=collection1
[junit4:junit4]   2> 19601 T3510 C213 P54861 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 19602 T3450 C212 P54848 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 19605 T3417 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19616 T3448 C212 P54848 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 19649 T3448 C212 P54848 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=3
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@549256d3; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@549256d3; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@549256d3; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_4,generation=4,filenames=[segments_4]
[junit4:junit4]   2> 19649 T3448 C212 P54848 oasc.SolrDeletionPolicy.updateCommits newest commit = 4[segments_4]
[junit4:junit4]   2> 19650 T3448 C212 P54848 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty1
[junit4:junit4]   2> 19650 T3448 C212 P54848 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 19651 T3448 C212 P54848 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 35
[junit4:junit4]   2> 19652 T3510 C213 P54861 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 19652 T3510 C213 P54861 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 19654 T3452 C212 P54848 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 19655 T3510 C213 P54861 oash.SnapPuller.fetchLatestIndex Master's generation: 4
[junit4:junit4]   2> 19655 T3510 C213 P54861 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 19655 T3510 C213 P54861 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 19657 T3452 C212 P54848 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty1
[junit4:junit4]   2> 19658 T3452 C212 P54848 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty1/index
[junit4:junit4]   2> 19658 T3452 C212 P54848 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=4&version=2} status=0 QTime=1 
[junit4:junit4]   2> 19659 T3510 C213 P54861 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 19660 T3510 C213 P54861 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty4/index.20130130225855155 forceNew:false
[junit4:junit4]   2> 19661 T3510 C213 P54861 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty4
[junit4:junit4]   2> 19662 T3510 C213 P54861 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty4/index.20130130225855155 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ded7710; maxCacheMB=48.0 maxMergeSizeMB=4.0)) fullCopy=true
[junit4:junit4]   2> 19665 T3452 C212 P54848 REQ /replication {file=segments_4&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=4} status=0 QTime=0 
[junit4:junit4]   2> 19666 T3510 C213 P54861 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty4/index
[junit4:junit4]   2> 19667 T3510 C213 P54861 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 19667 T3510 C213 P54861 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130130225855155
[junit4:junit4]   2> 19668 T3510 C213 P54861 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty4
[junit4:junit4]   2> 19669 T3510 C213 P54861 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty4
[junit4:junit4]   2> 19669 T3510 C213 P54861 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty4/index/ new=./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty4/index.20130130225855155
[junit4:junit4]   2> 19672 T3510 C213 P54861 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty4/index.20130130225855155 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ded7710; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_4,generation=4,filenames=[segments_4]
[junit4:junit4]   2> 19673 T3510 C213 P54861 oasc.SolrDeletionPolicy.updateCommits newest commit = 4[segments_4]
[junit4:junit4]   2> 19674 T3510 C213 P54861 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359619115501/jetty4
[junit4:junit4]   2> 19675 T3510 C213 P54861 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 19675 T3510 C213 P54861 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 19675 T3510 C213 P54861 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 19676 T3510 C213 P54861 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Luc

[...truncated too long message...]

eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  25) Thread[id=890, name=searcherExecutor-878-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  26) Thread[id=891, name=HashSessionScavenger-19, state=TIMED_WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at java.lang.Object.wait(Native Method)
[junit4:junit4]   2> 	        at java.util.TimerThread.mainLoop(Timer.java:552)
[junit4:junit4]   2> 	        at java.util.TimerThread.run(Timer.java:505)
[junit4:junit4]   2> 	  27) Thread[id=1275, name=searcherExecutor-1048-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  28) Thread[id=921, name=searcherExecutor-898-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  29) Thread[id=902, name=TEST-BasicDistributedZkTest.testDistribSearch-seed#[4B69744A3D9F79F0]-SendThread(localhost:22004), state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:82)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
[junit4:junit4]   2> 	  30) Thread[id=877, name=qtp2142436616-877 Acceptor0 SocketConnector@127.0.0.1:57725, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at java.net.PlainSocketImpl.socketAccept(Native Method)
[junit4:junit4]   2> 	        at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
[junit4:junit4]   2> 	        at java.net.ServerSocket.implAccept(ServerSocket.java:522)
[junit4:junit4]   2> 	        at java.net.ServerSocket.accept(ServerSocket.java:490)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.bio.SocketConnector.accept(SocketConnector.java:117)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> NOTE: test params are: codec=Lucene41: {n_f1=PostingsFormat(name=Memory doPackFST= false), foo_b=Pulsing41(freqCutoff=16 minBlockSize=97 maxBlockSize=206), cat=PostingsFormat(name=Lucene41WithOrds), foo_d=PostingsFormat(name=SimpleText), foo_f=Pulsing41(freqCutoff=16 minBlockSize=97 maxBlockSize=206), n_tl1=PostingsFormat(name=Memory doPackFST= false), n_d1=PostingsFormat(name=Lucene41WithOrds), rnd_b=PostingsFormat(name=SimpleText), intDefault=PostingsFormat(name=Memory doPackFST= false), n_td1=PostingsFormat(name=Memory doPackFST= false), timestamp=PostingsFormat(name=Lucene41WithOrds), id=PostingsFormat(name=SimpleText), range_facet_sl=PostingsFormat(name=SimpleText), range_facet_si=PostingsFormat(name=Lucene41WithOrds), oddField_s=PostingsFormat(name=SimpleText), sequence_i=PostingsFormat(name=SimpleText), name=PostingsFormat(name=SimpleText), foo_i=PostingsFormat(name=Memory doPackFST= false), regex_dup_B_s=PostingsFormat(name=Lucene41WithOrds), multiDefault=PostingsFormat(name=Lucene41WithOrds), n_tf1=PostingsFormat(name=Lucene41WithOrds), n_dt1=PostingsFormat(name=Memory doPackFST= false), genre_s=PostingsFormat(name=SimpleText), author_t=PostingsFormat(name=Memory doPackFST= false), n_ti1=PostingsFormat(name=SimpleText), range_facet_l=PostingsFormat(name=Lucene41WithOrds), text=Pulsing41(freqCutoff=16 minBlockSize=97 maxBlockSize=206), _version_=PostingsFormat(name=Memory doPackFST= false), val_i=PostingsFormat(name=SimpleText), SubjectTerms_mfacet=PostingsFormat(name=Memory doPackFST= false), series_t=PostingsFormat(name=Memory doPackFST= false), a_t=PostingsFormat(name=Lucene41WithOrds), n_tdt1=PostingsFormat(name=Memory doPackFST= false), regex_dup_A_s=Pulsing41(freqCutoff=16 minBlockSize=97 maxBlockSize=206), price=Pulsing41(freqCutoff=16 minBlockSize=97 maxBlockSize=206), other_tl1=PostingsFormat(name=Memory doPackFST= false), n_l1=PostingsFormat(name=Lucene41WithOrds), a_si=PostingsFormat(name=Memory doPackFST= false), inStock=Pulsing41(freqCutoff=16 minBlockSize=97 maxBlockSize=206)}, sim=DefaultSimilarity, locale=cs, timezone=Indian/Mauritius
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=31,free=277120056,total=501219328
[junit4:junit4]   2> NOTE: All tests run in this JVM: [DirectUpdateHandlerOptimizeTest, BadComponentTest, AlternateDirectoryTest, JsonLoaderTest, ShardRoutingCustomTest, TestFastOutputStream, TestXIncludeConfig, CSVRequestHandlerTest, AutoCommitTest, TestBinaryResponseWriter, TestLuceneMatchVersion, TestNumberUtils, BinaryUpdateRequestHandlerTest, TermsComponentTest, CoreAdminHandlerTest, TestReload, PreAnalyzedFieldTest, DateMathParserTest, DistributedTermsComponentTest, TestPerFieldSimilarity, ChaosMonkeySafeLeaderTest, LoggingHandlerTest, NumericFieldsTest, ChaosMonkeyNothingIsSafeTest, CollectionsAPIDistributedZkTest, ScriptEngineTest, TestLFUCache, ZkControllerTest, TestSuggestSpellingConverter, TestLMJelinekMercerSimilarityFactory, SearchHandlerTest, TestJmxMonitoredMap, TestCharFilters, TestJoin, BasicDistributedZkTest]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=BasicDistributedZkTest -Dtests.seed=4B69744A3D9F79F0 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=cs -Dtests.timezone=Indian/Mauritius -Dtests.file.encoding=US-ASCII
[junit4:junit4] ERROR   0.00s J0 | BasicDistributedZkTest (suite) <<<
[junit4:junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([4B69744A3D9F79F0]:0)
[junit4:junit4] Completed on J0 in 7223.90s, 1 test, 2 errors <<< FAILURES!

[...truncated 87 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:346: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/common-build.xml:428: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:1176: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:841: There were test failures: 247 suites, 1032 tests, 1 suite-level error, 1 error, 1 failure, 143 ignored (4 assumptions)

Total time: 143 minutes 50 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-trunk-java7 - Build # 3711 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-java7/3711/

4 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderTest

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest:     1) Thread[id=4585, name=Thread-1944-EventThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.ZkController.waitForLeaderToSeeDownState(ZkController.java:1241)         at org.apache.solr.cloud.ZkController.registerAllCoresAsDown(ZkController.java:304)         at org.apache.solr.cloud.ZkController.access$100(ZkController.java:85)         at org.apache.solr.cloud.ZkController$1.command(ZkController.java:193)         at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:117)         at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:46)         at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:91)         at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.ChaosMonkeySafeLeaderTest: 
   1) Thread[id=4585, name=Thread-1944-EventThread, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]
        at java.lang.Thread.sleep(Native Method)
        at org.apache.solr.cloud.ZkController.waitForLeaderToSeeDownState(ZkController.java:1241)
        at org.apache.solr.cloud.ZkController.registerAllCoresAsDown(ZkController.java:304)
        at org.apache.solr.cloud.ZkController.access$100(ZkController.java:85)
        at org.apache.solr.cloud.ZkController$1.command(ZkController.java:193)
        at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:117)
        at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:46)
        at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:91)
        at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
	at __randomizedtesting.SeedInfo.seed([3B2ECE55067E5F9]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=4585, name=Thread-1944-EventThread, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest]         at java.net.PlainSocketImpl.socketConnect(Native Method)         at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)         at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)         at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)         at java.net.Socket.connect(Socket.java:579)         at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:123)         at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:148)         at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:150)         at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:121)         at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:575)         at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:425)         at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)         at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)         at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:732)         at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:353)         at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181)         at org.apache.solr.cloud.ZkController.waitForLeaderToSeeDownState(ZkController.java:1236)         at org.apache.solr.cloud.ZkController.registerAllCoresAsDown(ZkController.java:304)         at org.apache.solr.cloud.ZkController.access$100(ZkController.java:85)         at org.apache.solr.cloud.ZkController$1.command(ZkController.java:193)         at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:117)         at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:46)         at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:91)         at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=4585, name=Thread-1944-EventThread, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest]
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:391)
        at java.net.Socket.connect(Socket.java:579)
        at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:123)
        at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:148)
        at org.apache.http.impl.conn.AbstractPoolEntry.open(AbstractPoolEntry.java:150)
        at org.apache.http.impl.conn.AbstractPooledConnAdapter.open(AbstractPooledConnAdapter.java:121)
        at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:575)
        at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:425)
        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:732)
        at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:353)
        at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181)
        at org.apache.solr.cloud.ZkController.waitForLeaderToSeeDownState(ZkController.java:1236)
        at org.apache.solr.cloud.ZkController.registerAllCoresAsDown(ZkController.java:304)
        at org.apache.solr.cloud.ZkController.access$100(ZkController.java:85)
        at org.apache.solr.cloud.ZkController$1.command(ZkController.java:193)
        at org.apache.solr.common.cloud.ConnectionManager$1.update(ConnectionManager.java:117)
        at org.apache.solr.common.cloud.DefaultConnectionStrategy.reconnect(DefaultConnectionStrategy.java:46)
        at org.apache.solr.common.cloud.ConnectionManager.process(ConnectionManager.java:91)
        at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
        at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
	at __randomizedtesting.SeedInfo.seed([3B2ECE55067E5F9]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([3B2ECE55067E5F9]:0)


FAILED:  org.apache.solr.cloud.BasicDistributedZkTest.testDistribSearch

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([3B2ECE55067E5F9]:0)




Build Log:
[...truncated 9343 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.ChaosMonkeySafeLeaderTest
[junit4:junit4]   2> 0 T4089 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4]   2> 5 T4089 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./solrtest-ChaosMonkeySafeLeaderTest-1360103291572
[junit4:junit4]   2> 7 T4089 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 7 T4090 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 10 T4090 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 11 T4090 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4]   2> 11 T4090 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4]   2> 11 T4090 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 12 T4090 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4]   2> 108 T4089 oasc.ZkTestServer.run start zk server on port:45907
[junit4:junit4]   2> 108 T4089 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45907 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@7c9bb633
[junit4:junit4]   2> 109 T4089 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 109 T4095 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45907. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 110 T4095 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45907, initiating session
[junit4:junit4]   2> 110 T4091 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:45908
[junit4:junit4]   2> 111 T4091 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:45908
[junit4:junit4]   2> 111 T4093 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 139 T4093 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cac793abb0000 with negotiated timeout 10000 for client /140.211.11.196:45908
[junit4:junit4]   2> 139 T4095 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45907, sessionid = 0x13cac793abb0000, negotiated timeout = 10000
[junit4:junit4]   2> 139 T4096 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7c9bb633 name:ZooKeeperConnection Watcher:127.0.0.1:45907 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 140 T4089 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 140 T4089 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 143 T4094 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13cac793abb0000
[junit4:junit4]   2> 145 T4096 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 145 T4089 oaz.ZooKeeper.close Session: 0x13cac793abb0000 closed
[junit4:junit4]   2> 145 T4091 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:45908 which had sessionid 0x13cac793abb0000
[junit4:junit4]   2> 146 T4089 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45907/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@36bd8461
[junit4:junit4]   2> 146 T4089 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 147 T4097 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45907. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 147 T4097 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45907, initiating session
[junit4:junit4]   2> 148 T4091 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:45909
[junit4:junit4]   2> 148 T4091 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:45909
[junit4:junit4]   2> 157 T4093 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cac793abb0001 with negotiated timeout 10000 for client /140.211.11.196:45909
[junit4:junit4]   2> 157 T4097 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45907, sessionid = 0x13cac793abb0001, negotiated timeout = 10000
[junit4:junit4]   2> 157 T4098 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@36bd8461 name:ZooKeeperConnection Watcher:127.0.0.1:45907/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 158 T4089 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 158 T4089 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 162 T4089 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 170 T4089 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 173 T4089 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 176 T4089 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 177 T4089 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 190 T4089 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 191 T4089 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 295 T4089 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 296 T4089 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 299 T4089 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 300 T4089 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 303 T4089 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 304 T4089 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 315 T4089 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 316 T4089 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 319 T4089 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 320 T4089 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 323 T4089 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 323 T4089 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 326 T4089 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 327 T4089 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 330 T4094 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13cac793abb0001
[junit4:junit4]   2> 337 T4098 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 337 T4091 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:45909 which had sessionid 0x13cac793abb0001
[junit4:junit4]   2> 337 T4089 oaz.ZooKeeper.close Session: 0x13cac793abb0001 closed
[junit4:junit4]   2> 510 T4089 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 515 T4089 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:45910
[junit4:junit4]   2> 516 T4089 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 516 T4089 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 517 T4089 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1360103291904
[junit4:junit4]   2> 517 T4089 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1360103291904/solr.xml
[junit4:junit4]   2> 518 T4089 oasc.CoreContainer.<init> New CoreContainer 1760950837
[junit4:junit4]   2> 518 T4089 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1360103291904/'
[junit4:junit4]   2> 519 T4089 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1360103291904/'
[junit4:junit4]   2> 576 T4089 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 577 T4089 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 577 T4089 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 578 T4089 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 578 T4089 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 579 T4089 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 579 T4089 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 580 T4089 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 580 T4089 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 581 T4089 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 603 T4089 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 623 T4089 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:45907/solr
[junit4:junit4]   2> 624 T4089 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 624 T4089 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45907 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@39d7ec24
[junit4:junit4]   2> 625 T4089 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 626 T4108 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45907. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 626 T4108 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45907, initiating session
[junit4:junit4]   2> 627 T4091 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:45911
[junit4:junit4]   2> 627 T4091 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:45911
[junit4:junit4]   2> 629 T4093 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cac793abb0002 with negotiated timeout 20000 for client /140.211.11.196:45911
[junit4:junit4]   2> 629 T4108 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45907, sessionid = 0x13cac793abb0002, negotiated timeout = 20000
[junit4:junit4]   2> 629 T4109 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@39d7ec24 name:ZooKeeperConnection Watcher:127.0.0.1:45907 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 629 T4089 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 631 T4094 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13cac793abb0002
[junit4:junit4]   2> 632 T4109 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 632 T4091 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:45911 which had sessionid 0x13cac793abb0002
[junit4:junit4]   2> 632 T4089 oaz.ZooKeeper.close Session: 0x13cac793abb0002 closed
[junit4:junit4]   2> 632 T4089 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 644 T4089 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45907/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@7ec372f5
[junit4:junit4]   2> 645 T4089 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 645 T4110 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45907. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 646 T4110 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45907, initiating session
[junit4:junit4]   2> 646 T4091 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:45912
[junit4:junit4]   2> 647 T4091 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:45912
[junit4:junit4]   2> 648 T4093 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cac793abb0003 with negotiated timeout 20000 for client /140.211.11.196:45912
[junit4:junit4]   2> 648 T4110 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45907, sessionid = 0x13cac793abb0003, negotiated timeout = 20000
[junit4:junit4]   2> 648 T4111 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7ec372f5 name:ZooKeeperConnection Watcher:127.0.0.1:45907/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 649 T4089 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 651 T4094 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cac793abb0003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 659 T4089 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 661 T4089 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45910_
[junit4:junit4]   2> 662 T4094 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cac793abb0003 type:delete cxid:0x7 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:45910_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:45910_
[junit4:junit4]   2> 663 T4089 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:45910_
[junit4:junit4]   2> 673 T4089 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 686 T4094 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cac793abb0003 type:delete cxid:0x15 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 687 T4089 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 690 T4089 oasc.Overseer.start Overseer (id=89135729316921347-127.0.0.1:45910_-n_0000000000) starting
[junit4:junit4]   2> 690 T4094 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cac793abb0003 type:create cxid:0x1a zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 697 T4094 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cac793abb0003 type:create cxid:0x1b zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 699 T4094 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cac793abb0003 type:create cxid:0x1c zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 705 T4094 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cac793abb0003 type:create cxid:0x1d zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 707 T4113 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 707 T4089 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 711 T4089 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 713 T4112 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 718 T4114 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1360103291904/collection1
[junit4:junit4]   2> 718 T4114 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 719 T4114 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 719 T4114 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 720 T4114 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1360103291904/collection1/'
[junit4:junit4]   2> 722 T4114 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1360103291904/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 722 T4114 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1360103291904/collection1/lib/README' to classloader
[junit4:junit4]   2> 780 T4114 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 855 T4114 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 956 T4114 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 966 T4114 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 1162 T3 oasc.CoreContainer.finalize SEVERE CoreContainer was not shutdown prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!  instance=649180823
[junit4:junit4]   2> 1163 T3 oasc.CoreContainer.finalize SEVERE CoreContainer was not shutdown prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!  instance=109314475
[junit4:junit4]   2> 1617 T4114 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1629 T4114 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1633 T4114 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1646 T4114 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1651 T4114 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1655 T4114 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1656 T4114 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1657 T4114 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-controljetty-1360103291904/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/control/data/
[junit4:junit4]   2> 1658 T4114 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1783206b
[junit4:junit4]   2> 1658 T4114 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 1659 T4114 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/control/data forceNew:false
[junit4:junit4]   2> 1660 T4114 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/control/data
[junit4:junit4]   2> 1660 T4114 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/control/data/index/
[junit4:junit4]   2> 1661 T4114 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1662 T4114 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/control/data/index forceNew:false
[junit4:junit4]   2> 1667 T4114 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5a609eee),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 1668 T4114 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 1669 T4114 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/control/data/index
[junit4:junit4]   2> 1669 T4114 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 1671 T4114 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1671 T4114 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 1672 T4114 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1672 T4114 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 1673 T4114 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1673 T4114 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1673 T4114 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1674 T4114 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1675 T4114 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1675 T4114 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 1676 T4114 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 1676 T4114 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1677 T4114 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 1677 T4114 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 1677 T4114 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 1678 T4114 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1678 T4114 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1679 T4114 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1679 T4114 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1680 T4114 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1681 T4114 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1681 T4114 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1682 T4114 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1682 T4114 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1683 T4114 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 1683 T4114 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 1695 T4114 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1700 T4114 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 1701 T4114 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 1703 T4114 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/control/data
[junit4:junit4]   2> 1705 T4114 oass.SolrIndexSearcher.<init> Opening Searcher@5ee1d38 main
[junit4:junit4]   2> 1706 T4114 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1706 T4114 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1707 T4114 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 1723 T4114 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 1779 T4115 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5ee1d38 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1786 T4114 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 1786 T4114 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 1786 T4094 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cac793abb0003 type:create cxid:0x50 zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 2216 T4112 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2217 T4112 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:45910_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:45910"}
[junit4:junit4]   2> 2217 T4112 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4]   2> 2218 T4112 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 2218 T4094 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cac793abb0003 type:create cxid:0x58 zxid:0x2c txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 2238 T4111 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 2790 T4114 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 2790 T4114 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:45910 collection:control_collection shard:shard1
[junit4:junit4]   2> 2791 T4114 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4]   2> 2808 T4094 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cac793abb0003 type:delete cxid:0x72 zxid:0x36 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4]   2> 2809 T4114 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 2809 T4094 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cac793abb0003 type:create cxid:0x73 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2811 T4114 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 2812 T4114 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 2812 T4114 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:45910/collection1/
[junit4:junit4]   2> 2812 T4114 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 2812 T4114 oasc.SyncStrategy.syncToMe http://127.0.0.1:45910/collection1/ has no replicas
[junit4:junit4]   2> 2813 T4114 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:45910/collection1/
[junit4:junit4]   2> 2813 T4114 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4]   2> 2817 T4094 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cac793abb0003 type:create cxid:0x7d zxid:0x3b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3742 T4112 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3760 T4111 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 3787 T4114 oasc.ZkController.register We are http://127.0.0.1:45910/collection1/ and leader is http://127.0.0.1:45910/collection1/
[junit4:junit4]   2> 3787 T4114 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:45910
[junit4:junit4]   2> 3787 T4114 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 3787 T4114 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 3788 T4114 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 3790 T4114 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3791 T4089 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 3792 T4089 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 3793 T4089 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 3807 T4089 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 3816 T4089 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45907/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@48201516
[junit4:junit4]   2> 3817 T4089 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 3817 T4116 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45907. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 3818 T4116 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45907, initiating session
[junit4:junit4]   2> 3818 T4091 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:45913
[junit4:junit4]   2> 3818 T4091 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:45913
[junit4:junit4]   2> 3820 T4093 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cac793abb0004 with negotiated timeout 10000 for client /140.211.11.196:45913
[junit4:junit4]   2> 3820 T4116 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45907, sessionid = 0x13cac793abb0004, negotiated timeout = 10000
[junit4:junit4]   2> 3820 T4117 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@48201516 name:ZooKeeperConnection Watcher:127.0.0.1:45907/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 3820 T4089 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 3821 T4089 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 3823 T4089 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 3992 T4089 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 3996 T4089 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:45914
[junit4:junit4]   2> 3997 T4089 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 3997 T4089 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 3998 T4089 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1360103295389
[junit4:junit4]   2> 3999 T4089 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1360103295389/solr.xml
[junit4:junit4]   2> 3999 T4089 oasc.CoreContainer.<init> New CoreContainer 368524678
[junit4:junit4]   2> 4000 T4089 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1360103295389/'
[junit4:junit4]   2> 4000 T4089 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1360103295389/'
[junit4:junit4]   2> 4057 T4089 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 4058 T4089 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 4058 T4089 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 4059 T4089 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 4060 T4089 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 4060 T4089 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 4061 T4089 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 4061 T4089 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 4062 T4089 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 4062 T4089 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 4082 T4089 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 4102 T4089 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:45907/solr
[junit4:junit4]   2> 4103 T4089 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 4104 T4089 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45907 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@4c1d65d1
[junit4:junit4]   2> 4105 T4089 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4105 T4127 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45907. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4106 T4127 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45907, initiating session
[junit4:junit4]   2> 4106 T4091 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:45915
[junit4:junit4]   2> 4106 T4091 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:45915
[junit4:junit4]   2> 4108 T4093 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cac793abb0005 with negotiated timeout 20000 for client /140.211.11.196:45915
[junit4:junit4]   2> 4108 T4127 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45907, sessionid = 0x13cac793abb0005, negotiated timeout = 20000
[junit4:junit4]   2> 4109 T4128 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4c1d65d1 name:ZooKeeperConnection Watcher:127.0.0.1:45907 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4109 T4089 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4110 T4094 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13cac793abb0005
[junit4:junit4]   2> 4123 T4128 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 4123 T4091 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:45915 which had sessionid 0x13cac793abb0005
[junit4:junit4]   2> 4123 T4089 oaz.ZooKeeper.close Session: 0x13cac793abb0005 closed
[junit4:junit4]   2> 4123 T4089 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 4133 T4089 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45907/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@277c0bf
[junit4:junit4]   2> 4134 T4089 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4135 T4129 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45907. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4135 T4129 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45907, initiating session
[junit4:junit4]   2> 4136 T4091 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:45916
[junit4:junit4]   2> 4136 T4091 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:45916
[junit4:junit4]   2> 4137 T4093 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cac793abb0006 with negotiated timeout 20000 for client /140.211.11.196:45916
[junit4:junit4]   2> 4137 T4129 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45907, sessionid = 0x13cac793abb0006, negotiated timeout = 20000
[junit4:junit4]   2> 4138 T4130 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@277c0bf name:ZooKeeperConnection Watcher:127.0.0.1:45907/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4138 T4089 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4139 T4094 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cac793abb0006 type:create cxid:0x1 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4140 T4094 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cac793abb0006 type:create cxid:0x2 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4142 T4089 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45914_
[junit4:junit4]   2> 4143 T4094 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cac793abb0006 type:delete cxid:0x4 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:45914_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:45914_
[junit4:junit4]   2> 4152 T4089 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:45914_
[junit4:junit4]   2> 4155 T4117 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 4155 T4111 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 4155 T4111 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 4158 T4089 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 4165 T4131 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1360103295389/collection1
[junit4:junit4]   2> 4166 T4131 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 4166 T4131 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 4167 T4131 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 4168 T4131 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1360103295389/collection1/'
[junit4:junit4]   2> 4169 T4131 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1360103295389/collection1/lib/README' to classloader
[junit4:junit4]   2> 4170 T4131 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1360103295389/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 4226 T4131 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 4326 T4131 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 4427 T4131 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 4436 T4131 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 5078 T4131 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 5087 T4131 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 5090 T4131 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 5100 T4131 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5103 T4131 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5106 T4131 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5107 T4131 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5108 T4131 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty1-1360103295389/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty1/
[junit4:junit4]   2> 5108 T4131 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1783206b
[junit4:junit4]   2> 5109 T4131 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 5109 T4131 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty1 forceNew:false
[junit4:junit4]   2> 5110 T4131 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty1
[junit4:junit4]   2> 5110 T4131 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty1/index/
[junit4:junit4]   2> 5110 T4131 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 5111 T4131 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty1/index forceNew:false
[junit4:junit4]   2> 5115 T4131 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@57ad8988),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 5116 T4131 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 5116 T4131 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty1/index
[junit4:junit4]   2> 5116 T4131 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 5118 T4131 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 5118 T4131 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 5119 T4131 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 5119 T4131 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 5119 T4131 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 5119 T4131 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 5120 T4131 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 5120 T4131 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 5121 T4131 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 5121 T4131 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 5122 T4131 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 5122 T4131 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 5123 T4131 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 5123 T4131 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 5123 T4131 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 5124 T4131 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 5124 T4131 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5125 T4131 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5125 T4131 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5126 T4131 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5126 T4131 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5127 T4131 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5127 T4131 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5128 T4131 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5128 T4131 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 5129 T4131 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 5137 T4131 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 5140 T4131 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 5141 T4131 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 5143 T4131 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty1
[junit4:junit4]   2> 5144 T4131 oass.SolrIndexSearcher.<init> Opening Searcher@4d8e3765 main
[junit4:junit4]   2> 5145 T4131 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 5145 T4131 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 5146 T4131 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 5151 T4131 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 5203 T4132 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4d8e3765 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 5209 T4131 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 5209 T4131 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 5265 T4112 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 5266 T4112 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:45910_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:45910"}
[junit4:junit4]   2> 5269 T4112 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"3",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:45914_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:45914"}
[junit4:junit4]   2> 5269 T4112 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 3
[junit4:junit4]   2> 5270 T4112 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 5281 T4130 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5281 T4111 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5281 T4117 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6211 T4131 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 6211 T4131 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:45914 collection:collection1 shard:shard1
[junit4:junit4]   2> 6212 T4131 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 6235 T4094 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cac793abb0006 type:delete cxid:0x43 zxid:0x5a txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 6236 T4131 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 6236 T4094 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cac793abb0006 type:create cxid:0x44 zxid:0x5b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6238 T4131 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 6239 T4131 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 6239 T4131 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:45914/collection1/
[junit4:junit4]   2> 6239 T4131 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 6240 T4131 oasc.SyncStrategy.syncToMe http://127.0.0.1:45914/collection1/ has no replicas
[junit4:junit4]   2> 6240 T4131 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:45914/collection1/
[junit4:junit4]   2> 6240 T4131 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 6244 T4094 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cac793abb0006 type:create cxid:0x4e zxid:0x5f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6794 T4112 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6813 T4130 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6813 T4117 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6813 T4111 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6858 T4131 oasc.ZkController.register We are http://127.0.0.1:45914/collection1/ and leader is http://127.0.0.1:45914/collection1/
[junit4:junit4]   2> 6858 T4131 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:45914
[junit4:junit4]   2> 6858 T4131 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 6858 T4131 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 6859 T4131 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 6861 T4131 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6862 T4089 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 6863 T4089 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 6863 T4089 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 7045 T4089 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 7049 T4089 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:45921
[junit4:junit4]   2> 7050 T4089 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 7050 T4089 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 7051 T4089 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1360103298444
[junit4:junit4]   2> 7051 T4089 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1360103298444/solr.xml
[junit4:junit4]   2> 7052 T4089 oasc.CoreContainer.<init> New CoreContainer 498757348
[junit4:junit4]   2> 7053 T4089 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1360103298444/'
[junit4:junit4]   2> 7053 T4089 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1360103298444/'
[junit4:junit4]   2> 7109 T4089 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 7110 T4089 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 7111 T4089 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 7111 T4089 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 7112 T4089 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 7112 T4089 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 7113 T4089 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 7113 T4089 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 7114 T4089 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 7114 T4089 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 7136 T4089 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 7156 T4089 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:45907/solr
[junit4:junit4]   2> 7157 T4089 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 7157 T4089 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45907 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@7833d74a
[junit4:junit4]   2> 7159 T4089 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 7159 T4142 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45907. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 7160 T4142 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45907, initiating session
[junit4:junit4]   2> 7160 T4091 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:45922
[junit4:junit4]   2> 7161 T4091 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:45922
[junit4:junit4]   2> 7162 T4093 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cac793abb0007 with negotiated timeout 20000 for client /140.211.11.196:45922
[junit4:junit4]   2> 7162 T4142 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45907, sessionid = 0x13cac793abb0007, negotiated timeout = 20000
[junit4:junit4]   2> 7163 T4143 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7833d74a name:ZooKeeperConnection Watcher:127.0.0.1:45907 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 7163 T4089 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 7164 T4094 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13cac793abb0007
[junit4:junit4]   2> 7174 T4143 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 7174 T4091 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:45922 which had sessionid 0x13cac793abb0007
[junit4:junit4]   2> 7174 T4089 oaz.ZooKeeper.close Session: 0x13cac793abb0007 closed
[junit4:junit4]   2> 7175 T4089 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 7185 T4089 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45907/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@38b721df
[junit4:junit4]   2> 7186 T4089 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 7186 T4144 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45907. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 7187 T4144 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45907, initiating session
[junit4:junit4]   2> 7187 T4091 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:45923
[junit4:junit4]   2> 7188 T4091 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:45923
[junit4:junit4]   2> 7189 T4093 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cac793abb0008 with negotiated timeout 20000 for client /140.211.11.196:45923
[junit4:junit4]   2> 7189 T4144 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45907, sessionid = 0x13cac793abb0008, negotiated timeout = 20000
[junit4:junit4]   2> 7189 T4145 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@38b721df name:ZooKeeperConnection Watcher:127.0.0.1:45907/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 7190 T4089 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 7191 T4094 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cac793abb0008 type:create cxid:0x1 zxid:0x6c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7192 T4094 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cac793abb0008 type:create cxid:0x2 zxid:0x6d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7194 T4089 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45921_
[junit4:junit4]   2> 7194 T4094 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cac793abb0008 type:delete cxid:0x4 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:45921_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:45921_
[junit4:junit4]   2> 7201 T4089 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:45921_
[junit4:junit4]   2> 7203 T4117 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7204 T4130 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7204 T4111 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7204 T4130 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 7204 T4111 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 7205 T4117 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7207 T4089 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 7214 T4146 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1360103298444/collection1
[junit4:junit4]   2> 7215 T4146 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 7215 T4146 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 7216 T4146 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 7217 T4146 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1360103298444/collection1/'
[junit4:junit4]   2> 7218 T4146 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1360103298444/collection1/lib/README' to classloader
[junit4:junit4]   2> 7219 T4146 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1360103298444/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 7276 T4146 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 7351 T4146 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 7452 T4146 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 7460 T4146 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 8112 T4146 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 8125 T4146 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 8129 T4146 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 8142 T4146 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8215 T4146 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8220 T4146 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8221 T4146 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8222 T4146 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty2-1360103298444/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty2/
[junit4:junit4]   2> 8223 T4146 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1783206b
[junit4:junit4]   2> 8223 T4146 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 8224 T4146 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty2 forceNew:false
[junit4:junit4]   2> 8225 T4146 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty2
[junit4:junit4]   2> 8225 T4146 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty2/index/
[junit4:junit4]   2> 8226 T4146 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 8227 T4146 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty2/index forceNew:false
[junit4:junit4]   2> 8232 T4146 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6be9e050),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 8233 T4146 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 8233 T4146 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty2/index
[junit4:junit4]   2> 8234 T4146 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 8236 T4146 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 8236 T4146 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 8237 T4146 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 8237 T4146 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 8238 T4146 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 8238 T4146 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 8239 T4146 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 8239 T4146 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 8240 T4146 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 8241 T4146 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 8241 T4146 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 8242 T4146 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 8242 T4146 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 8243 T4146 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 8243 T4146 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 8244 T4146 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 8244 T4146 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8245 T4146 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8245 T4146 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8246 T4146 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8247 T4146 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8247 T4146 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8248 T4146 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8248 T4146 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8249 T4146 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 8249 T4146 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 8262 T4146 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 8267 T4146 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 8268 T4146 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 8271 T4146 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty2
[junit4:junit4]   2> 8273 T4146 oass.SolrIndexSearcher.<init> Opening Searcher@c45adc6 main
[junit4:junit4]   2> 8274 T4146 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 8274 T4146 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 8275 T4146 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 8282 T4146 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 8319 T4112 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8320 T4112 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"3",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:45914_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:45914"}
[junit4:junit4]   2> 8330 T4111 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8330 T4117 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8330 T4130 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8330 T4145 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8337 T4147 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@c45adc6 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 8343 T4146 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 8343 T4146 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 9842 T4112 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9843 T4112 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"3",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:45921_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:45921"}
[junit4:junit4]   2> 9843 T4112 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4:junit4]   2> 9844 T4112 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 9888 T4111 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9888 T4130 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9888 T4117 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9888 T4145 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10347 T4146 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 10347 T4146 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:45921 collection:collection1 shard:shard2
[junit4:junit4]   2> 10348 T4146 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4]   2> 10364 T4094 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cac793abb0008 type:delete cxid:0x44 zxid:0x7d txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
[junit4:junit4]   2> 10366 T4146 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 10366 T4094 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cac793abb0008 type:create cxid:0x45 zxid:0x7e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 10375 T4146 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 10375 T4146 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 10376 T4146 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:45921/collection1/
[junit4:junit4]   2> 10376 T4146 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 10376 T4146 oasc.SyncStrategy.syncToMe http://127.0.0.1:45921/collection1/ has no replicas
[junit4:junit4]   2> 10377 T4146 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:45921/collection1/
[junit4:junit4]   2> 10377 T4146 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4]   2> 10380 T4094 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cac793abb0008 type:create cxid:0x4e zxid:0x81 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 11392 T4112 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11411 T4145 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11412 T4130 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11412 T4111 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11411 T4117 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11452 T4146 oasc.ZkController.register We are http://127.0.0.1:45921/collection1/ and leader is http://127.0.0.1:45921/collection1/
[junit4:junit4]   2> 11452 T4146 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:45921
[junit4:junit4]   2> 11452 T4146 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 11452 T4146 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 11453 T4146 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 11454 T4146 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11455 T4089 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 11456 T4089 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 11457 T4089 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 11638 T4089 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 11641 T4089 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:45926
[junit4:junit4]   2> 11642 T4089 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 11643 T4089 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 11643 T4089 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1360103303037
[junit4:junit4]   2> 11644 T4089 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1360103303037/solr.xml
[junit4:junit4]   2> 11644 T4089 oasc.CoreContainer.<init> New CoreContainer 1893353221
[junit4:junit4]   2> 11645 T4089 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1360103303037/'
[junit4:junit4]   2> 11645 T4089 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1360103303037/'
[junit4:junit4]   2> 11702 T4089 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 11703 T4089 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 11703 T4089 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 11704 T4089 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 11704 T4089 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 11705 T4089 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 11705 T4089 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 11706 T4089 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 11706 T4089 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 11707 T4089 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 11726 T4089 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 11746 T4089 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:45907/solr
[junit4:junit4]   2> 11747 T4089 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 11748 T4089 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45907 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@5451636e
[junit4:junit4]   2> 11749 T4089 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 11749 T4157 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45907. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 11750 T4157 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45907, initiating session
[junit4:junit4]   2> 11750 T4091 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:45927
[junit4:junit4]   2> 11750 T4091 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:45927
[junit4:junit4]   2> 11753 T4093 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cac793abb0009 with negotiated timeout 20000 for client /140.211.11.196:45927
[junit4:junit4]   2> 11753 T4157 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45907, sessionid = 0x13cac793abb0009, negotiated timeout = 20000
[junit4:junit4]   2> 11753 T4158 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5451636e name:ZooKeeperConnection Watcher:127.0.0.1:45907 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 11753 T4089 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 11755 T4094 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13cac793abb0009
[junit4:junit4]   2> 11770 T4158 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 11770 T4091 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:45927 which had sessionid 0x13cac793abb0009
[junit4:junit4]   2> 11770 T4089 oaz.ZooKeeper.close Session: 0x13cac793abb0009 closed
[junit4:junit4]   2> 11771 T4089 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 11781 T4089 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45907/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@455055a3
[junit4:junit4]   2> 11782 T4089 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 11782 T4159 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45907. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 11783 T4159 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45907, initiating session
[junit4:junit4]   2> 11783 T4091 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:45928
[junit4:junit4]   2> 11783 T4091 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:45928
[junit4:junit4]   2> 11784 T4093 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cac793abb000a with negotiated timeout 20000 for client /140.211.11.196:45928
[junit4:junit4]   2> 11785 T4159 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45907, sessionid = 0x13cac793abb000a, negotiated timeout = 20000
[junit4:junit4]   2> 11785 T4160 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@455055a3 name:ZooKeeperConnection Watcher:127.0.0.1:45907/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 11785 T4089 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 11786 T4094 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cac793abb000a type:create cxid:0x1 zxid:0x8e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 11788 T4094 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cac793abb000a type:create cxid:0x2 zxid:0x8f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 11795 T4089 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45926_
[junit4:junit4]   2> 11795 T4094 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cac793abb000a type:delete cxid:0x4 zxid:0x90 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:45926_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:45926_
[junit4:junit4]   2> 11797 T4089 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:45926_
[junit4:junit4]   2> 11799 T4117 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11799 T4145 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 11800 T4111 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 11800 T4145 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11800 T4130 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 11800 T4111 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11800 T4130 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11801 T4117 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 11809 T4089 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 11816 T4161 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1360103303037/collection1
[junit4:junit4]   2> 11816 T4161 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 11817 T4161 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 11817 T4161 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 11819 T4161 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1360103303037/collection1/'
[junit4:junit4]   2> 11820 T4161 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1360103303037/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 11820 T4161 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1360103303037/collection1/lib/README' to classloader
[junit4:junit4]   2> 11876 T4161 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 11951 T4161 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 12052 T4161 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 12060 T4161 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 12699 T4161 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 12710 T4161 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 12714 T4161 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 12727 T4161 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 12732 T4161 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 12736 T4161 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 12738 T4161 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 12739 T4161 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty3-1360103303037/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty3/
[junit4:junit4]   2> 12739 T4161 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1783206b
[junit4:junit4]   2> 12740 T4161 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 12740 T4161 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty3 forceNew:false
[junit4:junit4]   2> 12741 T4161 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty3
[junit4:junit4]   2> 12741 T4161 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty3/index/
[junit4:junit4]   2> 12742 T4161 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 12743 T4161 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty3/index forceNew:false
[junit4:junit4]   2> 12748 T4161 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3a925c15),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 12748 T4161 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 12749 T4161 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty3/index
[junit4:junit4]   2> 12750 T4161 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 12752 T4161 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 12752 T4161 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 12752 T4161 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 12753 T4161 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 12753 T4161 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 12754 T4161 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 12754 T4161 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 12755 T4161 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 12755 T4161 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 12756 T4161 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 12756 T4161 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 12757 T4161 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 12757 T4161 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 12758 T4161 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 12758 T4161 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 12759 T4161 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 12759 T4161 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12760 T4161 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12760 T4161 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12761 T4161 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12761 T4161 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12762 T4161 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12762 T4161 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12763 T4161 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12763 T4161 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 12764 T4161 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 12776 T4161 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 12780 T4161 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 12781 T4161 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 12784 T4161 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty3
[junit4:junit4]   2> 12786 T4161 oass.SolrIndexSearcher.<init> Opening Searcher@41f062b main
[junit4:junit4]   2> 12787 T4161 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 12787 T4161 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 12787 T4161 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 12794 T4161 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 12844 T4162 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@41f062b main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 12849 T4161 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 12849 T4161 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 12917 T4112 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12918 T4112 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"3",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:45921_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:45921"}
[junit4:junit4]   2> 12921 T4112 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"3",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:45926_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:45926"}
[junit4:junit4]   2> 12922 T4112 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4:junit4]   2> 12922 T4112 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard3
[junit4:junit4]   2> 12933 T4130 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12933 T4145 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12933 T4160 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12933 T4111 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12933 T4117 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13851 T4161 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 13851 T4161 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:45926 collection:collection1 shard:shard3
[junit4:junit4]   2> 13852 T4161 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard3/election
[junit4:junit4]   2> 13868 T4094 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cac793abb000a type:delete cxid:0x42 zxid:0x9e txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard3 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard3
[junit4:junit4]   2> 13869 T4161 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 13870 T4094 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cac793abb000a type:create cxid:0x43 zxid:0x9f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 13872 T4161 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 13872 T4161 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 13873 T4161 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:45926/collection1/
[junit4:junit4]   2> 13873 T4161 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 13873 T4161 oasc.SyncStrategy.syncToMe http://127.0.0.1:45926/collection1/ has no replicas
[junit4:junit4]   2> 13873 T4161 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:45926/collection1/
[junit4:junit4]   2> 13874 T4161 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard3
[junit4:junit4]   2> 13877 T4094 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cac793abb000a type:create cxid:0x4c zxid:0xa2 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 14447 T4112 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14506 T4117 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14506 T4145 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14506 T4111 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14506 T4160 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14506 T4130 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14541 T4161 oasc.ZkController.register We are http://127.0.0.1:45926/collection1/ and leader is http://127.0.0.1:45926/collection1/
[junit4:junit4]   2> 14541 T4161 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:45926
[junit4:junit4]   2> 14541 T4161 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 14542 T4161 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 14542 T4161 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 14543 T4161 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14544 T4089 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 14545 T4089 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 14546 T4089 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 14727 T4089 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 14730 T4089 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:45929
[junit4:junit4]   2> 14731 T4089 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 14732 T4089 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 14732 T4089 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1360103306126
[junit4:junit4]   2> 14733 T4089 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1360103306126/solr.xml
[junit4:junit4]   2> 14734 T4089 oasc.CoreContainer.<init> New CoreContainer 491567217
[junit4:junit4]   2> 14734 T4089 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1360103306126/'
[junit4:junit4]   2> 14735 T4089 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1360103306126/'
[junit4:junit4]   2> 14791 T4089 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 14792 T4089 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 14792 T4089 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 14793 T4089 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 14794 T4089 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 14794 T4089 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 14795 T4089 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 14795 T4089 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 14796 T4089 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 14796 T4089 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 14818 T4089 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 14838 T4089 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:45907/solr
[junit4:junit4]   2> 14839 T4089 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 14839 T4089 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45907 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@7c62ae70
[junit4:junit4]   2> 14840 T4089 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 14841 T4172 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45907. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 14841 T4172 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45907, initiating session
[junit4:junit4]   2> 14842 T4091 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:45930
[junit4:junit4]   2> 14842 T4091 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:45930
[junit4:junit4]   2> 14870 T4093 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cac793abb000b with negotiated timeout 20000 for client /140.211.11.196:45930
[junit4:junit4]   2> 14870 T4172 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45907, sessionid = 0x13cac793abb000b, negotiated timeout = 20000
[junit4:junit4]   2> 14871 T4173 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7c62ae70 name:ZooKeeperConnection Watcher:127.0.0.1:45907 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 14871 T4089 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 14872 T4094 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13cac793abb000b
[junit4:junit4]   2> 14881 T4173 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 14881 T4091 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:45930 which had sessionid 0x13cac793abb000b
[junit4:junit4]   2> 14881 T4089 oaz.ZooKeeper.close Session: 0x13cac793abb000b closed
[junit4:junit4]   2> 14882 T4089 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 14893 T4089 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45907/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@386b587a
[junit4:junit4]   2> 14894 T4089 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 14895 T4174 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45907. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 14895 T4174 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45907, initiating session
[junit4:junit4]   2> 14896 T4091 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:45931
[junit4:junit4]   2> 14896 T4091 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:45931
[junit4:junit4]   2> 14897 T4093 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cac793abb000c with negotiated timeout 20000 for client /140.211.11.196:45931
[junit4:junit4]   2> 14897 T4174 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45907, sessionid = 0x13cac793abb000c, negotiated timeout = 20000
[junit4:junit4]   2> 14898 T4175 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@386b587a name:ZooKeeperConnection Watcher:127.0.0.1:45907/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 14898 T4089 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 14899 T4094 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cac793abb000c type:create cxid:0x1 zxid:0xaf txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 14901 T4094 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cac793abb000c type:create cxid:0x2 zxid:0xb0 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 14902 T4089 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45929_
[junit4:junit4]   2> 14903 T4094 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cac793abb000c type:delete cxid:0x4 zxid:0xb1 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:45929_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:45929_
[junit4:junit4]   2> 14911 T4089 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:45929_
[junit4:junit4]   2> 14913 T4117 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14913 T4130 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 14914 T4111 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 14914 T4160 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 14914 T4145 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 14914 T4111 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14914 T4130 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14914 T4145 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14914 T4160 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14915 T4117 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 14917 T4089 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 14924 T4176 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1360103306126/collection1
[junit4:junit4]   2> 14925 T4176 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 14925 T4176 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 14925 T4176 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 14927 T4176 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1360103306126/collection1/'
[junit4:junit4]   2> 14928 T4176 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1360103306126/collection1/lib/README' to classloader
[junit4:junit4]   2> 14929 T4176 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1360103306126/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 14984 T4176 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 15080 T4176 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 15181 T4176 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 15189 T4176 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 15837 T4176 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 15850 T4176 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 15854 T4176 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 15867 T4176 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 15872 T4176 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 15877 T4176 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 15878 T4176 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 15879 T4176 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty4-1360103306126/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty4/
[junit4:junit4]   2> 15880 T4176 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1783206b
[junit4:junit4]   2> 15880 T4176 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 15881 T4176 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty4 forceNew:false
[junit4:junit4]   2> 15882 T4176 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty4
[junit4:junit4]   2> 15882 T4176 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty4/index/
[junit4:junit4]   2> 15883 T4176 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 15884 T4176 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty4/index forceNew:false
[junit4:junit4]   2> 15889 T4176 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3b31b6c2),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 15889 T4176 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 15890 T4176 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty4/index
[junit4:junit4]   2> 15891 T4176 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 15893 T4176 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 15893 T4176 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 15894 T4176 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 15894 T4176 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 15895 T4176 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 15895 T4176 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 15895 T4176 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 15896 T4176 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 15897 T4176 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 15897 T4176 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 15898 T4176 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 15898 T4176 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 15899 T4176 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 15899 T4176 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 15900 T4176 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 15900 T4176 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 15901 T4176 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15902 T4176 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15902 T4176 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15903 T4176 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15903 T4176 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15904 T4176 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15904 T4176 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15905 T4176 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15906 T4176 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 15906 T4176 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 15919 T4176 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 15924 T4176 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 15925 T4176 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 15928 T4176 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty4
[junit4:junit4]   2> 15929 T4176 oass.SolrIndexSearcher.<init> Opening Searcher@499ac127 main
[junit4:junit4]   2> 15930 T4176 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 15931 T4176 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 15931 T4176 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 15938 T4176 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 15985 T4177 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@499ac127 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 15990 T4176 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 15991 T4176 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 16043 T4112 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16044 T4112 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"3",
[junit4:junit4]   2> 	  "shard":"shard3",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:45926_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:45926"}
[junit4:junit4]   2> 16048 T4112 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"3",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:45929_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:45929"}
[junit4:junit4]   2> 16048 T4112 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4:junit4]   2> 16049 T4112 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 16060 T4160 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16060 T4111 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16060 T4175 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16060 T4130 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16060 T4117 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16060 T4145 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16993 T4176 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 16993 T4176 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:45929 collection:collection1 shard:shard1
[junit4:junit4]   2> 16996 T4176 oasc.ZkController.register We are http://127.0.0.1:45929/collection1/ and leader is http://127.0.0.1:45914/collection1/
[junit4:junit4]   2> 16997 T4176 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:45929
[junit4:junit4]   2> 16997 T4176 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 16997 T4176 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C214 name=collection1 org.apache.solr.core.SolrCore@4810ee3e url=http://127.0.0.1:45929/collection1 node=127.0.0.1:45929_ C214_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:45929_, base_url=http://127.0.0.1:45929}
[junit4:junit4]   2> 16998 T4178 C214 P45929 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 16998 T4176 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17000 T4178 C214 P45929 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 17000 T4178 C214 P45929 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 17000 T4178 C214 P45929 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 17000 T4089 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 17001 T4089 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 17002 T4178 C214 P45929 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 17002 T4089 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 17193 T4089 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 17196 T4089 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:52676
[junit4:junit4]   2> 17197 T4089 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 17198 T4089 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 17198 T4089 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1360103308591
[junit4:junit4]   2> 17199 T4089 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1360103308591/solr.xml
[junit4:junit4]   2> 17199 T4089 oasc.CoreContainer.<init> New CoreContainer 1267772678
[junit4:junit4]   2> 17200 T4089 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1360103308591/'
[junit4:junit4]   2> 17200 T4089 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1360103308591/'
[junit4:junit4]   2> 17257 T4089 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 17257 T4089 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 17258 T4089 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 17258 T4089 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 17259 T4089 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 17260 T4089 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 17260 T4089 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 17261 T4089 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 17261 T4089 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 17262 T4089 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 17283 T4089 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 17303 T4089 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:45907/solr
[junit4:junit4]   2> 17304 T4089 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 17305 T4089 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45907 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@1306eea4
[junit4:junit4]   2> 17306 T4089 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 17306 T4188 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45907. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 17307 T4188 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45907, initiating session
[junit4:junit4]   2> 17307 T4091 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:30555
[junit4:junit4]   2> 17308 T4091 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:30555
[junit4:junit4]   2> 17310 T4093 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cac793abb000d with negotiated timeout 20000 for client /140.211.11.196:30555
[junit4:junit4]   2> 17310 T4188 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45907, sessionid = 0x13cac793abb000d, negotiated timeout = 20000
[junit4:junit4]   2> 17310 T4189 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1306eea4 name:ZooKeeperConnection Watcher:127.0.0.1:45907 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 17311 T4089 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 17312 T4094 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13cac793abb000d
[junit4:junit4]   2> 17324 T4189 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 17324 T4091 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:30555 which had sessionid 0x13cac793abb000d
[junit4:junit4]   2> 17324 T4089 oaz.ZooKeeper.close Session: 0x13cac793abb000d closed
[junit4:junit4]   2> 17325 T4089 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 17337 T4089 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45907/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@3a92ab6
[junit4:junit4]   2> 17338 T4089 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 17338 T4190 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45907. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 17339 T4190 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45907, initiating session
[junit4:junit4]   2> 17339 T4091 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:56643
[junit4:junit4]   2> 17339 T4091 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:56643
[junit4:junit4]   2> 17341 T4093 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cac793abb000e with negotiated timeout 20000 for client /140.211.11.196:56643
[junit4:junit4]   2> 17341 T4190 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45907, sessionid = 0x13cac793abb000e, negotiated timeout = 20000
[junit4:junit4]   2> 17341 T4191 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3a92ab6 name:ZooKeeperConnection Watcher:127.0.0.1:45907/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 17341 T4089 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 17342 T4094 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cac793abb000e type:create cxid:0x1 zxid:0xc1 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 17344 T4094 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cac793abb000e type:create cxid:0x2 zxid:0xc2 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 17351 T4089 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52676_
[junit4:junit4]   2> 17352 T4094 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cac793abb000e type:delete cxid:0x4 zxid:0xc3 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:52676_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:52676_
[junit4:junit4]   2> 17354 T4089 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52676_
[junit4:junit4]   2> 17356 T4117 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17356 T4160 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 17357 T4145 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 17357 T4130 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 17356 T4175 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 17357 T4145 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 17357 T4160 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 17357 T4111 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 17358 T4117 oascc.ZkStateReader$3.process Updating live nodes... (6)
[junit4:junit4]   2> 17357 T4175 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 17357 T4130 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 17358 T4111 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 17360 T4089 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 17367 T4192 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1360103308591/collection1
[junit4:junit4]   2> 17367 T4192 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 17368 T4192 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 17368 T4192 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 17370 T4192 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1360103308591/collection1/'
[junit4:junit4]   2> 17371 T4192 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1360103308591/collection1/lib/README' to classloader
[junit4:junit4]   2> 17372 T4192 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1360103308591/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 17428 T4192 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 17505 T4192 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 17574 T4112 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17574 T4112 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"3",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:45929_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:45929"}
[junit4:junit4]   2> 17590 T4130 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 17590 T4160 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 17591 T4175 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 17591 T4111 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 17591 T4145 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 17590 T4117 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 17606 T4192 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 17606 T4191 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 17612 T4192 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 18269 T4192 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 18281 T4192 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 18286 T4192 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 18298 T4192 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 18304 T4192 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 18308 T4192 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 18309 T4192 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 18311 T4192 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty5-1360103308591/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty5/
[junit4:junit4]   2> 18311 T4192 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1783206b
[junit4:junit4]   2> 18312 T4192 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 18313 T4192 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty5 forceNew:false
[junit4:junit4]   2> 18313 T4192 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty5
[junit4:junit4]   2> 18314 T4192 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty5/index/
[junit4:junit4]   2> 18314 T4192 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty5/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 18315 T4192 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty5/index forceNew:false
[junit4:junit4]   2> 18321 T4192 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty5/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2e979ab7),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 18321 T4192 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 18322 T4192 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty5/index
[junit4:junit4]   2> 18322 T4192 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 18325 T4192 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 18325 T4192 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 18326 T4192 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 18326 T4192 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 18327 T4192 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 18327 T4192 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 18327 T4192 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 18328 T4192 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 18329 T4192 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 18329 T4192 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 18330 T4192 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 18330 T4192 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 18331 T4192 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 18331 T4192 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 18332 T4192 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 18332 T4192 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 18333 T4192 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 18334 T4192 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 18334 T4192 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 18335 T4192 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 18335 T4192 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 18336 T4192 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 18336 T4192 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 18337 T4192 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 18338 T4192 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 18338 T4192 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 18351 T4192 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 18356 T4192 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 18357 T4192 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 18360 T4192 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-1360103291572/jetty5
[junit4:junit4]   2> 18361 T4192 oass.SolrIndexSearcher.<init> Opening Searcher@4e6fcd38 main
[junit4:junit4]   2> 18362 T4192 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 18363 T4192 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 18363 T4192 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 18371 T4192 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 18421 T4193 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4e6fcd38 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 18426 T4192 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 18427 T4192 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 19095 T4112 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19096 T4112 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=3 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"3",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:52676_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:52676"}
[junit4:junit4]   2> 19096 T4112 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=3
[junit4:junit4]   2> 19096 T4112 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 19114 T4130 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 19114 T4191 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 19114 T4117 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 19114 T4160 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 19114 T4175 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 19114 T4145 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 19114 T4111 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 19429 T4192 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 19429 T4192 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:52676 collection:collection1 shard:shard2
[junit4:junit4]   2> 19433 T4192 oasc.ZkController.register We are http://127.0.0.1:52676/collection1/ and leader is http://127.0.0.1:45921/collection1/
[junit4:junit4]   2> 19433 T4192 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:52676
[junit4:junit4]   2> 19433 T4192 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 19433 T4192 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C215 name=collection1 org.apache.solr.core.SolrCore@1b7e4c28 url=http://127.0.0.1:52676/collection1 node=127.0.0.1:52676_ C215_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:52676_, base_url=http://127.0.0.1:52676}
[junit4:junit4]   2> 19434 T4194 C215 P52676 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 19434 T4192 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19435 T4194 C215 P52676 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 19436 T4194 C215 P52676 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 19436 T4194 C215 P52676 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 19436 T4089 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 19437 T4089 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 19438 T4194 C215 P52676 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 19438 T4089 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 19621 T4089 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 19624 T4089 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:31031
[junit4:junit4]   2> 19625 T4089 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 19626 T4089 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 19626 T4089 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1360103311020
[junit4:junit4]   2> 19627 T4089 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1360103311020/solr.xml
[junit4:junit4]   2> 19627 T4089 oasc.CoreContainer.<init> New CoreContainer 509902003
[junit4:junit4]   2> 19628 T4089 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1360103311020/'
[junit4:junit4]   2> 19629 T4089 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.ChaosMonkeySafeLeaderTest-jetty6-1360103311020/'
[junit4:junit4]   2> 19685 T4089 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 19686 T4089 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 19686 T4089 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 19687 T4089 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 19687 T4089 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 19688 T4089 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 19688 T4089 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 19689 T4089 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 19689 T4089 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 19690 T4089 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 19709 T4089 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 19730 T4089 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:45907/solr
[junit4:junit4]   2> 19731 T4089 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 19732 T4089 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45907 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@48c5a02b
[junit4:junit4]   2> 19733 T4089 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 19734 T4204 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45907. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 19734 T4204 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45907, initiating session
[junit4:junit4]   2> 19734 T4091 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:34943
[junit4:junit4]   2> 19735 T4091 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:34943
[junit4:junit4]   2> 19762 T4093 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cac793abb000f with negotiated timeout 20000 for client /140.211.11.196:34943
[junit4:junit4]   2> 19762 T4204 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45907, sessionid = 0x13cac793abb000f, negotiated timeout = 20000
[junit4:junit4]   2> 19763 T4205 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@48c5a02b name:ZooKeeperConnection Watcher:127.0.0.1:45907 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 19763 T4089 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 19764 T4094 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13cac793abb000f
[junit4:junit4]   2> 19765 T4205 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 19765 T4091 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:34943 which had sessionid 0x13cac793abb000f
[junit4:junit4]   2> 19765 T4089 oaz.ZooKeeper.close Session: 0x13cac793abb000f closed
[junit4:junit4]   2> 19766 T4089 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 19775 T4089 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45907/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@5682d37e
[junit4:junit4]   2> 19777 T4089 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 19777 T4206 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45907. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 19777 T4206 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45907, initiating session
[junit4:junit4]   2> 19778 T4091 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:62483
[junit4:junit4]   2> 19778 T4091 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:62483
[junit4:junit4]   2> 19779 T4093 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cac793abb0010 with negotiated timeout 20000 for client /140.211.11.196:62483
[junit4:junit4]   2> 19779 T4206 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45907, sessionid = 0x13cac793abb0010, negotiated timeout = 20000
[junit4:junit4]   2> 19780 T4207 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5682d37e name:ZooKeeperConnection Watcher:127.0.0.1:45907/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 19780 T4089 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 19781 T4094 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cac793abb0010 type:create cxid:0x1 zxid:0xd4 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 19782 T4094 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cac793abb0010 type:create cxid:0x2 zxid:0xd5 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 19791 T4089 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:31031_
[junit4:junit4]   2> 19791 T4094 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cac793abb0010 type:delete cxid:0x4 zxid:0xd6 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:31031_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:31031_
[junit4:junit4]   2> 19792 T4089 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:31031_
[junit4:junit4]   2> 19795 T4191 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 19795 T4117 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 6)
[junit4:junit4]   2> 19796 T4130 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4]   2> 19796 T4111 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4]   2> 19796 T4160 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4]   2> 19796 T4130 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 7)
[junit4:junit4]   2> 19796 T4145 oascc.ZkStateReader$3.process Updating live nodes... (7)
[junit4:junit4]   2> 19796 T4175 

[...truncated too long message...]

ool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  18) Thread[id=548, name=searcherExecutor-456-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  19) Thread[id=530, name=TEST-BasicDistributedZkTest.testDistribSearch-seed#[3B2ECE55067E5F9]-SendThread(localhost:43773), state=TIMED_WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at java.lang.Thread.sleep(Native Method)
[junit4:junit4]   2> 	        at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:86)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:937)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:993)
[junit4:junit4]   2> 	  20) Thread[id=535, name=qtp596863905-535 Acceptor0 SocketConnector@127.0.0.1:43799, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at java.net.PlainSocketImpl.socketAccept(Native Method)
[junit4:junit4]   2> 	        at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
[junit4:junit4]   2> 	        at java.net.ServerSocket.implAccept(ServerSocket.java:522)
[junit4:junit4]   2> 	        at java.net.ServerSocket.accept(ServerSocket.java:490)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.bio.SocketConnector.accept(SocketConnector.java:117)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  21) Thread[id=575, name=TEST-BasicDistributedZkTest.testDistribSearch-seed#[3B2ECE55067E5F9]-SendThread(localhost:43773), state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.nio.ch.IOUtil.drain(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.KQueueSelectorImpl.updateSelectedKeys(KQueueSelectorImpl.java:167)
[junit4:junit4]   2> 	        at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:108)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
[junit4:junit4]   2> 	  22) Thread[id=550, name=HashSessionScavenger-4, state=TIMED_WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at java.lang.Object.wait(Native Method)
[junit4:junit4]   2> 	        at java.util.TimerThread.mainLoop(Timer.java:552)
[junit4:junit4]   2> 	        at java.util.TimerThread.run(Timer.java:505)
[junit4:junit4]   2> 	  23) Thread[id=551, name=qtp1688655239-551 Acceptor0 SocketConnector@127.0.0.1:43805, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at java.net.PlainSocketImpl.socketAccept(Native Method)
[junit4:junit4]   2> 	        at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
[junit4:junit4]   2> 	        at java.net.ServerSocket.implAccept(ServerSocket.java:522)
[junit4:junit4]   2> 	        at java.net.ServerSocket.accept(ServerSocket.java:490)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.bio.SocketConnector.accept(SocketConnector.java:117)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  24) Thread[id=534, name=HashSessionScavenger-3, state=TIMED_WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at java.lang.Object.wait(Native Method)
[junit4:junit4]   2> 	        at java.util.TimerThread.mainLoop(Timer.java:552)
[junit4:junit4]   2> 	        at java.util.TimerThread.run(Timer.java:505)
[junit4:junit4]   2> NOTE: test params are: codec=Lucene42: {n_f1=PostingsFormat(name=Asserting), foo_b=PostingsFormat(name=Lucene41WithOrds), foo_d=PostingsFormat(name=MockRandom), foo_f=PostingsFormat(name=Lucene41WithOrds), n_tl1=PostingsFormat(name=Asserting), n_d1=PostingsFormat(name=SimpleText), rnd_b=PostingsFormat(name=MockRandom), intDefault=PostingsFormat(name=SimpleText), n_td1=PostingsFormat(name=Asserting), timestamp=PostingsFormat(name=SimpleText), id=PostingsFormat(name=MockRandom), range_facet_sl=PostingsFormat(name=Lucene41WithOrds), range_facet_si=PostingsFormat(name=Asserting), oddField_s=PostingsFormat(name=Lucene41WithOrds), foo_i=PostingsFormat(name=Asserting), multiDefault=PostingsFormat(name=Asserting), n_tf1=PostingsFormat(name=SimpleText), n_dt1=PostingsFormat(name=Asserting), n_ti1=PostingsFormat(name=MockRandom), range_facet_l=PostingsFormat(name=SimpleText), text=PostingsFormat(name=Lucene41WithOrds), _version_=PostingsFormat(name=SimpleText), val_i=PostingsFormat(name=MockRandom), SubjectTerms_mfacet=PostingsFormat(name=SimpleText), a_t=PostingsFormat(name=SimpleText), n_tdt1=PostingsFormat(name=SimpleText), other_tl1=PostingsFormat(name=SimpleText), n_l1=PostingsFormat(name=SimpleText), a_si=PostingsFormat(name=Asserting)}, sim=DefaultSimilarity, locale=es_PR, timezone=Europe/Samara
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=26,free=83003344,total=244318208
[junit4:junit4]   2> NOTE: All tests run in this JVM: [TestJoin, SuggesterWFSTTest, TestNumberUtils, TestXIncludeConfig, TestWordDelimiterFilterFactory, HighlighterConfigTest, UpdateRequestProcessorFactoryTest, TestZkChroot, SOLR749Test, TestCoreContainer, AnalysisAfterCoreReloadTest, TestIBSimilarityFactory, TestFuzzyAnalyzedSuggestions, ScriptEngineTest, TestPostingsSolrHighlighter, TestFastOutputStream, SpellPossibilityIteratorTest, PreAnalyzedFieldTest, TestCharFilters, TestSolr4Spatial, ChaosMonkeyNothingIsSafeTest, TestReload, FileBasedSpellCheckerTest, TestSearchPerf, WordBreakSolrSpellCheckerTest, IndexBasedSpellCheckerTest, DistanceFunctionTest, ClusterStateUpdateTest, MoreLikeThisHandlerTest, UniqFieldsUpdateProcessorFactoryTest, ExternalFileFieldSortTest, SolrCoreCheckLockOnStartupTest, SortByFunctionTest, LoggingHandlerTest, AutoCommitTest, CSVRequestHandlerTest, MinimalSchemaTest, TestSuggestSpellingConverter, JSONWriterTest, TestStressVersions, TestGroupingSearch, TestCollationField, SolrPluginUtilsTest, DefaultValueUpdateProcessorTest, TestExtendedDismaxParser, TestElisionMultitermQuery, TestDefaultSimilarityFactory, TestFiltering, TestPerFieldSimilarity, SolrRequestParserTest, TestFaceting, TestIndexingPerformance, SpatialFilterTest, TestSystemIdResolver, TestDocumentBuilder, NotRequiredUniqueKeyTest, SpellCheckCollatorTest, TestLMJelinekMercerSimilarityFactory, BasicDistributedZkTest]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=BasicDistributedZkTest -Dtests.seed=3B2ECE55067E5F9 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es_PR -Dtests.timezone=Europe/Samara -Dtests.file.encoding=ISO-8859-1
[junit4:junit4] ERROR   0.00s J0 | BasicDistributedZkTest (suite) <<<
[junit4:junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([3B2ECE55067E5F9]:0)
[junit4:junit4] Completed on J0 in 7223.55s, 1 test, 2 errors <<< FAILURES!

[...truncated 12 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:346: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/common-build.xml:432: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:1176: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:841: There were test failures: 249 suites, 1038 tests, 3 suite-level errors, 1 error, 78 ignored (4 assumptions)

Total time: 148 minutes 17 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-trunk-java7 - Build # 3710 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-java7/3710/

2 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([B5C653350651FA91]:0)


FAILED:  org.apache.solr.cloud.BasicDistributedZkTest.testDistribSearch

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([B5C653350651FA91]:0)




Build Log:
[...truncated 9523 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
[junit4:junit4]   2> 0 T397 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /s
[junit4:junit4]   2> 5 T397 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZkTest-1360067984389
[junit4:junit4]   2> 7 T397 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 7 T398 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 10 T398 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 10 T398 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4]   2> 10 T398 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4]   2> 11 T398 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 12 T398 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4]   2> 108 T397 oasc.ZkTestServer.run start zk server on port:40554
[junit4:junit4]   2> 108 T397 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:40554 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@59a6a49
[junit4:junit4]   2> 109 T397 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 109 T403 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:40554. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 110 T403 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:40554, initiating session
[junit4:junit4]   2> 110 T399 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:28395
[junit4:junit4]   2> 111 T399 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:28395
[junit4:junit4]   2> 111 T401 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 114 T401 oazs.ZooKeeperServer.finishSessionInit Established session 0x13caa5e7c0b0000 with negotiated timeout 10000 for client /140.211.11.196:28395
[junit4:junit4]   2> 114 T403 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:40554, sessionid = 0x13caa5e7c0b0000, negotiated timeout = 10000
[junit4:junit4]   2> 115 T404 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@59a6a49 name:ZooKeeperConnection Watcher:127.0.0.1:40554 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 115 T397 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 115 T397 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 135 T402 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13caa5e7c0b0000
[junit4:junit4]   2> 136 T404 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 136 T397 oaz.ZooKeeper.close Session: 0x13caa5e7c0b0000 closed
[junit4:junit4]   2> 136 T399 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:28395 which had sessionid 0x13caa5e7c0b0000
[junit4:junit4]   2> 137 T397 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:40554/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@14ca1ff7
[junit4:junit4]   2> 138 T397 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 138 T405 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:40554. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 138 T405 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:40554, initiating session
[junit4:junit4]   2> 139 T399 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:58493
[junit4:junit4]   2> 139 T399 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:58493
[junit4:junit4]   2> 141 T401 oazs.ZooKeeperServer.finishSessionInit Established session 0x13caa5e7c0b0001 with negotiated timeout 10000 for client /140.211.11.196:58493
[junit4:junit4]   2> 141 T405 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:40554, sessionid = 0x13caa5e7c0b0001, negotiated timeout = 10000
[junit4:junit4]   2> 141 T406 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14ca1ff7 name:ZooKeeperConnection Watcher:127.0.0.1:40554/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 141 T397 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 142 T397 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 147 T397 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 151 T397 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 159 T397 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 164 T397 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 165 T397 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 178 T397 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 179 T397 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 285 T397 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 285 T397 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 290 T397 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 291 T397 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 295 T397 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 296 T397 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 301 T397 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 301 T397 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 307 T397 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 308 T397 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 312 T397 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 313 T397 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 318 T397 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 318 T397 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 323 T402 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13caa5e7c0b0001
[junit4:junit4]   2> 324 T406 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 324 T397 oaz.ZooKeeper.close Session: 0x13caa5e7c0b0001 closed
[junit4:junit4]   2> 324 T399 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:58493 which had sessionid 0x13caa5e7c0b0001
[junit4:junit4]   2> 1077 T397 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 1311 T397 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:15496
[junit4:junit4]   2> 1348 T397 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 1348 T397 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1348 T397 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360067984708
[junit4:junit4]   2> 1349 T397 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360067984708/solr.xml
[junit4:junit4]   2> 1349 T397 oasc.CoreContainer.<init> New CoreContainer 1334230517
[junit4:junit4]   2> 1350 T397 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360067984708/'
[junit4:junit4]   2> 1350 T397 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360067984708/'
[junit4:junit4]   2> 1387 T397 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 1387 T397 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 1387 T397 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 1388 T397 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 1388 T397 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 1388 T397 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 1389 T397 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 1389 T397 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 1389 T397 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 1390 T397 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 1407 T397 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 1422 T397 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:40554/solr
[junit4:junit4]   2> 1422 T397 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 1423 T397 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:40554 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@7b420664
[junit4:junit4]   2> 1436 T397 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1437 T417 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:40554. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 1438 T417 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:40554, initiating session
[junit4:junit4]   2> 1439 T399 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:64157
[junit4:junit4]   2> 1439 T399 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:64157
[junit4:junit4]   2> 1440 T401 oazs.ZooKeeperServer.finishSessionInit Established session 0x13caa5e7c0b0002 with negotiated timeout 20000 for client /140.211.11.196:64157
[junit4:junit4]   2> 1440 T417 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:40554, sessionid = 0x13caa5e7c0b0002, negotiated timeout = 20000
[junit4:junit4]   2> 1441 T418 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7b420664 name:ZooKeeperConnection Watcher:127.0.0.1:40554 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1441 T397 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1442 T402 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13caa5e7c0b0002
[junit4:junit4]   2> 1443 T399 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:64157 which had sessionid 0x13caa5e7c0b0002
[junit4:junit4]   2> 1443 T418 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 1443 T397 oaz.ZooKeeper.close Session: 0x13caa5e7c0b0002 closed
[junit4:junit4]   2> 1444 T397 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 1454 T397 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:40554/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@78ffa42a
[junit4:junit4]   2> 1454 T397 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1455 T419 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:40554. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 1455 T419 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:40554, initiating session
[junit4:junit4]   2> 1455 T399 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:13674
[junit4:junit4]   2> 1456 T399 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:13674
[junit4:junit4]   2> 1457 T401 oazs.ZooKeeperServer.finishSessionInit Established session 0x13caa5e7c0b0003 with negotiated timeout 20000 for client /140.211.11.196:13674
[junit4:junit4]   2> 1457 T419 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:40554, sessionid = 0x13caa5e7c0b0003, negotiated timeout = 20000
[junit4:junit4]   2> 1458 T420 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@78ffa42a name:ZooKeeperConnection Watcher:127.0.0.1:40554/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1458 T397 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1460 T402 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caa5e7c0b0003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 1467 T397 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 1470 T397 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:15496_s
[junit4:junit4]   2> 1471 T402 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caa5e7c0b0003 type:delete cxid:0x7 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:15496_s Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:15496_s
[junit4:junit4]   2> 1472 T397 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:15496_s
[junit4:junit4]   2> 1476 T397 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 1492 T402 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caa5e7c0b0003 type:delete cxid:0x15 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 1493 T397 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 1496 T397 oasc.Overseer.start Overseer (id=89133415425310723-127.0.0.1:15496_s-n_0000000000) starting
[junit4:junit4]   2> 1497 T402 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caa5e7c0b0003 type:create cxid:0x1a zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 1503 T402 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caa5e7c0b0003 type:create cxid:0x1b zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 1504 T402 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caa5e7c0b0003 type:create cxid:0x1c zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 1511 T402 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caa5e7c0b0003 type:create cxid:0x1d zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 1512 T422 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 1513 T397 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 1516 T397 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 1522 T423 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360067984708/collection1
[junit4:junit4]   2> 1522 T421 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 1522 T423 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 1523 T423 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 1523 T423 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 1525 T423 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360067984708/collection1/'
[junit4:junit4]   2> 1526 T423 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360067984708/collection1/lib/README' to classloader
[junit4:junit4]   2> 1527 T423 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360067984708/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 1580 T423 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1651 T423 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1752 T423 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 1758 T423 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 2394 T423 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 2407 T423 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 2411 T423 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 2436 T423 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2441 T423 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2447 T423 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2449 T423 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2450 T423 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360067984708/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/control/data/
[junit4:junit4]   2> 2451 T423 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ad875c
[junit4:junit4]   2> 2452 T423 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 2489 T423 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/control/data forceNew:false
[junit4:junit4]   2> 2490 T423 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/control/data
[junit4:junit4]   2> 2490 T423 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/control/data/index/
[junit4:junit4]   2> 2491 T423 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 2492 T423 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/control/data/index forceNew:false
[junit4:junit4]   2> 2498 T423 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@19bc3c3c; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 2499 T423 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 2500 T423 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/control/data/index
[junit4:junit4]   2> 2500 T423 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 2502 T423 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 2503 T423 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 2503 T423 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 2503 T423 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 2504 T423 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 2504 T423 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 2505 T423 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 2505 T423 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 2506 T423 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 2506 T423 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 2507 T423 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 2507 T423 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 2508 T423 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 2508 T423 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 2508 T423 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 2509 T423 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 2509 T423 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2510 T423 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2510 T423 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2511 T423 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2511 T423 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2512 T423 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2512 T423 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2513 T423 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2513 T423 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 2514 T423 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 2527 T423 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 2531 T423 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 2533 T423 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 2535 T423 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/control/data
[junit4:junit4]   2> 2537 T423 oass.SolrIndexSearcher.<init> Opening Searcher@c914bfb main
[junit4:junit4]   2> 2538 T423 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 2539 T423 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 2539 T423 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 2558 T423 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 2614 T424 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@c914bfb main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 2620 T423 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 2620 T423 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 2621 T402 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caa5e7c0b0003 type:create cxid:0x50 zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 3028 T421 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3029 T421 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:15496_s",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:15496/s"}
[junit4:junit4]   2> 3029 T421 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4]   2> 3034 T421 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 3035 T402 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caa5e7c0b0003 type:create cxid:0x58 zxid:0x2c txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 3054 T420 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 3634 T423 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 3634 T423 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:15496/s collection:control_collection shard:shard1
[junit4:junit4]   2> 3635 T423 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4]   2> 3659 T402 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caa5e7c0b0003 type:delete cxid:0x72 zxid:0x36 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4]   2> 3660 T423 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 3661 T402 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caa5e7c0b0003 type:create cxid:0x73 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3671 T423 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 3672 T423 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 3672 T423 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:15496/s/collection1/
[junit4:junit4]   2> 3672 T423 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 3673 T423 oasc.SyncStrategy.syncToMe http://127.0.0.1:15496/s/collection1/ has no replicas
[junit4:junit4]   2> 3673 T423 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:15496/s/collection1/
[junit4:junit4]   2> 3673 T423 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4]   2> 3680 T402 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caa5e7c0b0003 type:create cxid:0x7d zxid:0x3b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4565 T421 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 4614 T420 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 4657 T423 oasc.ZkController.register We are http://127.0.0.1:15496/s/collection1/ and leader is http://127.0.0.1:15496/s/collection1/
[junit4:junit4]   2> 4657 T423 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:15496/s
[junit4:junit4]   2> 4657 T423 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 4657 T423 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 4658 T423 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 4661 T423 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 4662 T397 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 4663 T397 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 4736 T397 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 4763 T397 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 4779 T397 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:40554/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@18755efe
[junit4:junit4]   2> 4781 T397 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4781 T425 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:40554. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4781 T425 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:40554, initiating session
[junit4:junit4]   2> 4782 T399 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:21448
[junit4:junit4]   2> 4782 T399 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:21448
[junit4:junit4]   2> 4809 T401 oazs.ZooKeeperServer.finishSessionInit Established session 0x13caa5e7c0b0004 with negotiated timeout 10000 for client /140.211.11.196:21448
[junit4:junit4]   2> 4810 T425 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:40554, sessionid = 0x13caa5e7c0b0004, negotiated timeout = 10000
[junit4:junit4]   2> 4810 T426 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@18755efe name:ZooKeeperConnection Watcher:127.0.0.1:40554/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4810 T397 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4812 T397 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 4817 T397 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 4988 T397 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 4991 T397 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:11927
[junit4:junit4]   2> 4992 T397 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 4993 T397 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 4993 T397 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360067989200
[junit4:junit4]   2> 4994 T397 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360067989200/solr.xml
[junit4:junit4]   2> 4994 T397 oasc.CoreContainer.<init> New CoreContainer 1566760852
[junit4:junit4]   2> 4995 T397 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360067989200/'
[junit4:junit4]   2> 4995 T397 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360067989200/'
[junit4:junit4]   2> 5051 T397 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 5051 T397 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 5052 T397 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 5052 T397 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 5053 T397 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 5053 T397 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 5053 T397 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 5054 T397 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 5054 T397 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 5055 T397 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 5077 T397 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 5096 T397 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:40554/solr
[junit4:junit4]   2> 5096 T397 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 5097 T397 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:40554 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@64bd76f4
[junit4:junit4]   2> 5098 T397 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 5098 T436 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:40554. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 5099 T436 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:40554, initiating session
[junit4:junit4]   2> 5099 T399 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:13340
[junit4:junit4]   2> 5100 T399 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:13340
[junit4:junit4]   2> 5102 T401 oazs.ZooKeeperServer.finishSessionInit Established session 0x13caa5e7c0b0005 with negotiated timeout 20000 for client /140.211.11.196:13340
[junit4:junit4]   2> 5102 T436 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:40554, sessionid = 0x13caa5e7c0b0005, negotiated timeout = 20000
[junit4:junit4]   2> 5102 T437 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@64bd76f4 name:ZooKeeperConnection Watcher:127.0.0.1:40554 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 5102 T397 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 5104 T402 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13caa5e7c0b0005
[junit4:junit4]   2> 5113 T437 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 5113 T399 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:13340 which had sessionid 0x13caa5e7c0b0005
[junit4:junit4]   2> 5113 T397 oaz.ZooKeeper.close Session: 0x13caa5e7c0b0005 closed
[junit4:junit4]   2> 5114 T397 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 5126 T397 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:40554/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@7b4afbd3
[junit4:junit4]   2> 5127 T397 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 5127 T438 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:40554. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 5128 T438 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:40554, initiating session
[junit4:junit4]   2> 5128 T399 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:27291
[junit4:junit4]   2> 5128 T399 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:27291
[junit4:junit4]   2> 5129 T401 oazs.ZooKeeperServer.finishSessionInit Established session 0x13caa5e7c0b0006 with negotiated timeout 20000 for client /140.211.11.196:27291
[junit4:junit4]   2> 5130 T438 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:40554, sessionid = 0x13caa5e7c0b0006, negotiated timeout = 20000
[junit4:junit4]   2> 5130 T439 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7b4afbd3 name:ZooKeeperConnection Watcher:127.0.0.1:40554/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 5130 T397 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 5131 T402 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caa5e7c0b0006 type:create cxid:0x1 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5133 T402 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caa5e7c0b0006 type:create cxid:0x2 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5141 T397 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:11927_s
[junit4:junit4]   2> 5141 T402 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caa5e7c0b0006 type:delete cxid:0x4 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:11927_s Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:11927_s
[junit4:junit4]   2> 5143 T397 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:11927_s
[junit4:junit4]   2> 5146 T420 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 5147 T426 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 5149 T420 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 5152 T397 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 5161 T440 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360067989200/collection1
[junit4:junit4]   2> 5161 T440 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 5162 T440 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 5162 T440 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 5164 T440 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360067989200/collection1/'
[junit4:junit4]   2> 5165 T440 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360067989200/collection1/lib/README' to classloader
[junit4:junit4]   2> 5166 T440 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360067989200/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 5230 T440 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 5302 T440 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 5403 T440 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 5409 T440 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 6050 T440 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 6062 T440 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 6066 T440 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 6089 T440 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 6095 T440 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 6101 T440 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 6103 T440 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 6105 T440 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360067989200/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty1/
[junit4:junit4]   2> 6105 T440 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ad875c
[junit4:junit4]   2> 6106 T440 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 6106 T440 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty1 forceNew:false
[junit4:junit4]   2> 6107 T440 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty1
[junit4:junit4]   2> 6107 T440 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty1/index/
[junit4:junit4]   2> 6108 T440 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 6109 T440 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty1/index forceNew:false
[junit4:junit4]   2> 6115 T440 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@59ce2c7; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 6116 T440 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 6117 T440 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty1/index
[junit4:junit4]   2> 6117 T440 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 6120 T440 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 6120 T440 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 6120 T440 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 6121 T440 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 6121 T440 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 6122 T440 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 6122 T440 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 6123 T440 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 6123 T440 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 6124 T440 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 6125 T440 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 6125 T440 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 6126 T440 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 6126 T440 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 6126 T440 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 6127 T440 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 6128 T440 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6128 T440 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6128 T421 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6129 T440 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6129 T440 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6130 T421 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:15496_s",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:15496/s"}
[junit4:junit4]   2> 6130 T440 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6131 T440 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6132 T440 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6132 T440 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6133 T440 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 6133 T440 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 6136 T426 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6136 T439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6136 T420 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6147 T440 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 6152 T440 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 6154 T440 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 6156 T440 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty1
[junit4:junit4]   2> 6158 T440 oass.SolrIndexSearcher.<init> Opening Searcher@268dc656 main
[junit4:junit4]   2> 6159 T440 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 6159 T440 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 6160 T440 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 6168 T440 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 6224 T441 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@268dc656 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 6230 T440 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 6231 T440 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 7644 T421 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 7645 T421 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:11927_s",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:11927/s"}
[junit4:junit4]   2> 7645 T421 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4]   2> 7646 T421 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 7652 T426 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7652 T420 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7652 T439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8235 T440 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 8235 T440 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:11927/s collection:collection1 shard:shard1
[junit4:junit4]   2> 8236 T440 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 8300 T402 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caa5e7c0b0006 type:delete cxid:0x45 zxid:0x5b txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 8332 T440 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 8333 T402 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caa5e7c0b0006 type:create cxid:0x46 zxid:0x5c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8336 T440 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 8336 T440 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 8336 T440 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:11927/s/collection1/
[junit4:junit4]   2> 8336 T440 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 8337 T440 oasc.SyncStrategy.syncToMe http://127.0.0.1:11927/s/collection1/ has no replicas
[junit4:junit4]   2> 8337 T440 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:11927/s/collection1/
[junit4:junit4]   2> 8337 T440 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 8343 T402 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caa5e7c0b0006 type:create cxid:0x50 zxid:0x60 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9171 T421 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9237 T426 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 9237 T439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 9237 T420 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 9264 T440 oasc.ZkController.register We are http://127.0.0.1:11927/s/collection1/ and leader is http://127.0.0.1:11927/s/collection1/
[junit4:junit4]   2> 9264 T440 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:11927/s
[junit4:junit4]   2> 9264 T440 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 9265 T440 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 9265 T440 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 9268 T440 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9269 T397 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 9270 T397 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 9270 T397 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 9457 T397 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 9460 T397 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:16649
[junit4:junit4]   2> 9461 T397 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 9461 T397 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 9462 T397 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360067993669
[junit4:junit4]   2> 9462 T397 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360067993669/solr.xml
[junit4:junit4]   2> 9463 T397 oasc.CoreContainer.<init> New CoreContainer 1968406234
[junit4:junit4]   2> 9463 T397 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360067993669/'
[junit4:junit4]   2> 9464 T397 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360067993669/'
[junit4:junit4]   2> 9519 T397 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 9520 T397 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 9520 T397 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 9521 T397 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 9521 T397 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 9522 T397 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 9522 T397 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 9523 T397 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 9523 T397 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 9524 T397 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 9547 T397 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 9566 T397 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:40554/solr
[junit4:junit4]   2> 9567 T397 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 9567 T397 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:40554 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@5d3ffe7f
[junit4:junit4]   2> 9568 T397 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 9569 T451 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:40554. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 9570 T451 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:40554, initiating session
[junit4:junit4]   2> 9570 T399 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:41751
[junit4:junit4]   2> 9570 T399 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:41751
[junit4:junit4]   2> 9572 T401 oazs.ZooKeeperServer.finishSessionInit Established session 0x13caa5e7c0b0007 with negotiated timeout 20000 for client /140.211.11.196:41751
[junit4:junit4]   2> 9572 T451 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:40554, sessionid = 0x13caa5e7c0b0007, negotiated timeout = 20000
[junit4:junit4]   2> 9573 T452 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5d3ffe7f name:ZooKeeperConnection Watcher:127.0.0.1:40554 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 9573 T397 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 9575 T402 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13caa5e7c0b0007
[junit4:junit4]   2> 9586 T452 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 9586 T399 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:41751 which had sessionid 0x13caa5e7c0b0007
[junit4:junit4]   2> 9586 T397 oaz.ZooKeeper.close Session: 0x13caa5e7c0b0007 closed
[junit4:junit4]   2> 9587 T397 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 9601 T397 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:40554/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@6e255b2c
[junit4:junit4]   2> 9602 T397 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 9603 T453 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:40554. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 9603 T453 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:40554, initiating session
[junit4:junit4]   2> 9603 T399 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:13517
[junit4:junit4]   2> 9604 T399 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:13517
[junit4:junit4]   2> 9607 T401 oazs.ZooKeeperServer.finishSessionInit Established session 0x13caa5e7c0b0008 with negotiated timeout 20000 for client /140.211.11.196:13517
[junit4:junit4]   2> 9607 T453 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:40554, sessionid = 0x13caa5e7c0b0008, negotiated timeout = 20000
[junit4:junit4]   2> 9607 T454 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6e255b2c name:ZooKeeperConnection Watcher:127.0.0.1:40554/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 9607 T397 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 9609 T402 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caa5e7c0b0008 type:create cxid:0x1 zxid:0x6d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9611 T402 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caa5e7c0b0008 type:create cxid:0x2 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9613 T397 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:16649_s
[junit4:junit4]   2> 9615 T402 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caa5e7c0b0008 type:delete cxid:0x4 zxid:0x6f txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:16649_s Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:16649_s
[junit4:junit4]   2> 9616 T397 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:16649_s
[junit4:junit4]   2> 9619 T426 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 9619 T420 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 9620 T439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 9622 T420 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 9622 T439 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 9622 T426 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 9630 T397 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 9638 T455 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360067993669/collection1
[junit4:junit4]   2> 9638 T455 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 9639 T455 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 9639 T455 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 9641 T455 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360067993669/collection1/'
[junit4:junit4]   2> 9643 T455 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360067993669/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 9643 T455 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360067993669/collection1/lib/README' to classloader
[junit4:junit4]   2> 9707 T455 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 9779 T455 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 9880 T455 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 9886 T455 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 10530 T455 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 10542 T455 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 10546 T455 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 10571 T455 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 10577 T455 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 10583 T455 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 10585 T455 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 10587 T455 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360067993669/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty2/
[junit4:junit4]   2> 10587 T455 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ad875c
[junit4:junit4]   2> 10588 T455 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 10589 T455 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty2 forceNew:false
[junit4:junit4]   2> 10589 T455 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty2
[junit4:junit4]   2> 10590 T455 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty2/index/
[junit4:junit4]   2> 10590 T455 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 10591 T455 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty2/index forceNew:false
[junit4:junit4]   2> 10598 T455 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6756223f; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 10598 T455 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 10599 T455 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty2/index
[junit4:junit4]   2> 10600 T455 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 10602 T455 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 10602 T455 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 10603 T455 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 10603 T455 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 10604 T455 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 10604 T455 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 10605 T455 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 10605 T455 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 10606 T455 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 10607 T455 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 10607 T455 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 10608 T455 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 10608 T455 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 10609 T455 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 10609 T455 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 10610 T455 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 10610 T455 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10611 T455 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10611 T455 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10612 T455 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10613 T455 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10613 T455 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10614 T455 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10614 T455 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10615 T455 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 10616 T455 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 10628 T455 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 10632 T455 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 10634 T455 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 10636 T455 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty2
[junit4:junit4]   2> 10638 T455 oass.SolrIndexSearcher.<init> Opening Searcher@313de952 main
[junit4:junit4]   2> 10639 T455 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 10639 T455 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 10640 T455 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 10647 T455 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 10700 T456 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@313de952 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 10706 T455 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 10706 T455 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 10752 T421 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 10753 T421 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:11927_s",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:11927/s"}
[junit4:junit4]   2> 10759 T421 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:16649_s",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:16649/s"}
[junit4:junit4]   2> 10759 T421 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 10759 T421 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 10771 T454 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10771 T426 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10771 T439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10771 T420 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11710 T455 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 11710 T455 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:16649/s collection:collection1 shard:shard2
[junit4:junit4]   2> 11711 T455 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4]   2> 11728 T402 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caa5e7c0b0008 type:delete cxid:0x42 zxid:0x7d txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
[junit4:junit4]   2> 11729 T455 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 11730 T402 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caa5e7c0b0008 type:create cxid:0x43 zxid:0x7e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 11740 T455 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 11741 T455 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 11741 T455 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:16649/s/collection1/
[junit4:junit4]   2> 11741 T455 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 11741 T455 oasc.SyncStrategy.syncToMe http://127.0.0.1:16649/s/collection1/ has no replicas
[junit4:junit4]   2> 11742 T455 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:16649/s/collection1/
[junit4:junit4]   2> 11742 T455 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4]   2> 11747 T402 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caa5e7c0b0008 type:create cxid:0x4c zxid:0x81 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12286 T421 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12306 T454 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12306 T439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12306 T420 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12306 T426 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12311 T455 oasc.ZkController.register We are http://127.0.0.1:16649/s/collection1/ and leader is http://127.0.0.1:16649/s/collection1/
[junit4:junit4]   2> 12311 T455 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:16649/s
[junit4:junit4]   2> 12311 T455 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 12311 T455 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 12312 T455 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 12316 T455 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12318 T397 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 12319 T397 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 12319 T397 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 12510 T397 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 12513 T397 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:21808
[junit4:junit4]   2> 12514 T397 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 12514 T397 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 12515 T397 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360067996721
[junit4:junit4]   2> 12515 T397 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360067996721/solr.xml
[junit4:junit4]   2> 12516 T397 oasc.CoreContainer.<init> New CoreContainer 1999324040
[junit4:junit4]   2> 12517 T397 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360067996721/'
[junit4:junit4]   2> 12517 T397 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360067996721/'
[junit4:junit4]   2> 12573 T397 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 12574 T397 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 12574 T397 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 12575 T397 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 12575 T397 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 12576 T397 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 12576 T397 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 12577 T397 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 12577 T397 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 12577 T397 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 12599 T397 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 12619 T397 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:40554/solr
[junit4:junit4]   2> 12619 T397 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 12620 T397 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:40554 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@60cfddc8
[junit4:junit4]   2> 12621 T397 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 12622 T466 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:40554. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 12623 T466 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:40554, initiating session
[junit4:junit4]   2> 12623 T399 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:35024
[junit4:junit4]   2> 12623 T399 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:35024
[junit4:junit4]   2> 12626 T401 oazs.ZooKeeperServer.finishSessionInit Established session 0x13caa5e7c0b0009 with negotiated timeout 20000 for client /140.211.11.196:35024
[junit4:junit4]   2> 12626 T466 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:40554, sessionid = 0x13caa5e7c0b0009, negotiated timeout = 20000
[junit4:junit4]   2> 12626 T467 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@60cfddc8 name:ZooKeeperConnection Watcher:127.0.0.1:40554 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 12626 T397 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 12628 T402 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13caa5e7c0b0009
[junit4:junit4]   2> 12640 T467 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 12640 T399 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:35024 which had sessionid 0x13caa5e7c0b0009
[junit4:junit4]   2> 12640 T397 oaz.ZooKeeper.close Session: 0x13caa5e7c0b0009 closed
[junit4:junit4]   2> 12641 T397 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 12652 T397 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:40554/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@7df4b72f
[junit4:junit4]   2> 12654 T397 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 12654 T468 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:40554. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 12654 T468 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:40554, initiating session
[junit4:junit4]   2> 12655 T399 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:63107
[junit4:junit4]   2> 12655 T399 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:63107
[junit4:junit4]   2> 12657 T401 oazs.ZooKeeperServer.finishSessionInit Established session 0x13caa5e7c0b000a with negotiated timeout 20000 for client /140.211.11.196:63107
[junit4:junit4]   2> 12657 T468 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:40554, sessionid = 0x13caa5e7c0b000a, negotiated timeout = 20000
[junit4:junit4]   2> 12657 T469 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7df4b72f name:ZooKeeperConnection Watcher:127.0.0.1:40554/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 12657 T397 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 12658 T402 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caa5e7c0b000a type:create cxid:0x1 zxid:0x8e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12660 T402 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caa5e7c0b000a type:create cxid:0x2 zxid:0x8f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12663 T397 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:21808_s
[junit4:junit4]   2> 12664 T402 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caa5e7c0b000a type:delete cxid:0x4 zxid:0x90 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:21808_s Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:21808_s
[junit4:junit4]   2> 12671 T397 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:21808_s
[junit4:junit4]   2> 12674 T426 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12674 T420 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12674 T439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12675 T454 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 12676 T454 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12677 T426 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 12677 T439 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 12677 T420 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 12680 T397 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 12688 T470 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360067996721/collection1
[junit4:junit4]   2> 12689 T470 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 12690 T470 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 12690 T470 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 12692 T470 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360067996721/collection1/'
[junit4:junit4]   2> 12693 T470 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360067996721/collection1/lib/README' to classloader
[junit4:junit4]   2> 12694 T470 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360067996721/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 12758 T470 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 12831 T470 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 12932 T470 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 12938 T470 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 13600 T470 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 13612 T470 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 13616 T470 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 13637 T470 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 13643 T470 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 13648 T470 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 13650 T470 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 13651 T470 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360067996721/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty3/
[junit4:junit4]   2> 13652 T470 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ad875c
[junit4:junit4]   2> 13652 T470 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 13653 T470 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty3 forceNew:false
[junit4:junit4]   2> 13654 T470 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty3
[junit4:junit4]   2> 13654 T470 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty3/index/
[junit4:junit4]   2> 13655 T470 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 13655 T470 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty3/index forceNew:false
[junit4:junit4]   2> 13662 T470 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@20a0faaf; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 13662 T470 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 13663 T470 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty3/index
[junit4:junit4]   2> 13664 T470 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 13666 T470 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 13666 T470 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 13667 T470 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 13667 T470 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 13668 T470 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 13668 T470 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 13668 T470 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 13669 T470 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 13670 T470 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 13670 T470 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 13671 T470 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 13671 T470 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 13672 T470 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 13672 T470 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 13673 T470 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 13673 T470 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 13674 T470 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13674 T470 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13675 T470 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13675 T470 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13676 T470 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13677 T470 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13677 T470 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13678 T470 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13678 T470 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 13679 T470 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 13691 T470 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 13695 T470 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 13697 T470 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 13699 T470 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty3
[junit4:junit4]   2> 13701 T470 oass.SolrIndexSearcher.<init> Opening Searcher@3246246f main
[junit4:junit4]   2> 13702 T470 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 13702 T470 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 13703 T470 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 13710 T470 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 13767 T471 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3246246f main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 13773 T470 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 13773 T470 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 13823 T421 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 13824 T421 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:16649_s",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:16649/s"}
[junit4:junit4]   2> 13844 T421 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:21808_s",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:21808/s"}
[junit4:junit4]   2> 13844 T421 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 13844 T421 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 13856 T469 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13856 T420 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13856 T439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13856 T426 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13856 T454 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14776 T470 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 14776 T470 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:21808/s collection:collection1 shard:shard1
[junit4:junit4]   2> 14785 T470 oasc.ZkController.register We are http://127.0.0.1:21808/s/collection1/ and leader is http://127.0.0.1:11927/s/collection1/
[junit4:junit4]   2> 14786 T470 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:21808/s
[junit4:junit4]   2> 14786 T470 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 14786 T470 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C87 name=collection1 org.apache.solr.core.SolrCore@33344bc url=http://127.0.0.1:21808/s/collection1 node=127.0.0.1:21808_s C87_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:21808_s, base_url=http://127.0.0.1:21808/s}
[junit4:junit4]   2> 14793 T472 C87 P21808 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 14794 T470 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14794 T472 C87 P21808 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 14795 T472 C87 P21808 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 14795 T472 C87 P21808 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 14795 T397 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 14796 T397 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 14797 T472 C87 P21808 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 14797 T397 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 15044 T397 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 15047 T397 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:37523
[junit4:junit4]   2> 15048 T397 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 15049 T397 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 15049 T397 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360067999203
[junit4:junit4]   2> 15049 T397 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360067999203/solr.xml
[junit4:junit4]   2> 15050 T397 oasc.CoreContainer.<init> New CoreContainer 633908158
[junit4:junit4]   2> 15051 T397 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360067999203/'
[junit4:junit4]   2> 15051 T397 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360067999203/'
[junit4:junit4]   2> 15131 T397 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 15132 T397 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 15132 T397 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 15133 T397 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 15133 T397 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 15134 T397 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 15134 T397 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 15135 T397 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 15135 T397 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 15136 T397 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 15161 T397 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 15196 T397 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:40554/solr
[junit4:junit4]   2> 15196 T397 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 15197 T397 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:40554 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@7361b79f
[junit4:junit4]   2> 15198 T397 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 15199 T482 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:40554. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 15200 T482 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:40554, initiating session
[junit4:junit4]   2> 15200 T399 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:45488
[junit4:junit4]   2> 15200 T399 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:45488
[junit4:junit4]   2> 15202 T401 oazs.ZooKeeperServer.finishSessionInit Established session 0x13caa5e7c0b000b with negotiated timeout 20000 for client /140.211.11.196:45488
[junit4:junit4]   2> 15202 T482 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:40554, sessionid = 0x13caa5e7c0b000b, negotiated timeout = 20000
[junit4:junit4]   2> 15203 T483 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7361b79f name:ZooKeeperConnection Watcher:127.0.0.1:40554 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 15203 T397 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 15205 T402 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13caa5e7c0b000b
[junit4:junit4]   2> 15221 T483 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 15221 T399 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:45488 which had sessionid 0x13caa5e7c0b000b
[junit4:junit4]   2> 15221 T397 oaz.ZooKeeper.close Session: 0x13caa5e7c0b000b closed
[junit4:junit4]   2> 15222 T397 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 15235 T397 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:40554/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@38ade9c4
[junit4:junit4]   2> 15236 T397 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 15236 T484 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:40554. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 15237 T484 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:40554, initiating session
[junit4:junit4]   2> 15237 T399 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:38068
[junit4:junit4]   2> 15238 T399 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:38068
[junit4:junit4]   2> 15239 T401 oazs.ZooKeeperServer.finishSessionInit Established session 0x13caa5e7c0b000c with negotiated timeout 20000 for client /140.211.11.196:38068
[junit4:junit4]   2> 15239 T484 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:40554, sessionid = 0x13caa5e7c0b000c, negotiated timeout = 20000
[junit4:junit4]   2> 15239 T485 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@38ade9c4 name:ZooKeeperConnection Watcher:127.0.0.1:40554/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 15240 T397 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 15241 T402 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caa5e7c0b000c type:create cxid:0x1 zxid:0xa0 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 15243 T402 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caa5e7c0b000c type:create cxid:0x2 zxid:0xa1 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 15250 T397 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37523_s
[junit4:junit4]   2> 15251 T402 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13caa5e7c0b000c type:delete cxid:0x4 zxid:0xa2 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:37523_s Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:37523_s
[junit4:junit4]   2> 15252 T397 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:37523_s
[junit4:junit4]   2> 15256 T469 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15256 T439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15256 T426 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15256 T420 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15257 T454 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 15257 T454 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15260 T439 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 15260 T469 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 15260 T420 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 15260 T426 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 15263 T397 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 15272 T486 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360067999203/collection1
[junit4:junit4]   2> 15272 T486 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 15273 T486 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 15273 T486 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 15275 T486 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360067999203/collection1/'
[junit4:junit4]   2> 15277 T486 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360067999203/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 15277 T486 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360067999203/collection1/lib/README' to classloader
[junit4:junit4]   2> 15371 T421 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 15372 T421 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:21808_s",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:21808/s"}
[junit4:junit4]   2> 15382 T486 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 15385 T469 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15385 T426 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15386 T420 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15386 T439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15385 T485 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15385 T454 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15465 T486 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 15566 T486 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 15590 T486 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 16358 T486 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 16369 T486 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 16373 T486 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 16391 T486 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 16407 T486 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 16411 T486 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 16413 T486 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 16415 T486 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360067999203/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty4/
[junit4:junit4]   2> 16415 T486 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ad875c
[junit4:junit4]   2> 16416 T486 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 16416 T486 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty4 forceNew:false
[junit4:junit4]   2> 16417 T486 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty4
[junit4:junit4]   2> 16417 T486 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty4/index/
[junit4:junit4]   2> 16418 T486 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 16419 T486 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty4/index forceNew:false
[junit4:junit4]   2> 16425 T486 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@37035dff; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 16425 T486 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 16426 T486 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty4/index
[junit4:junit4]   2> 16426 T486 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 16429 T486 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 16429 T486 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 16429 T486 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 16430 T486 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 16430 T486 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 16430 T486 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 16431 T486 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 16431 T486 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 16432 T486 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 16433 T486 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 16433 T486 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 16434 T486 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 16434 T486 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 16434 T486 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 16435 T486 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 16435 T486 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 16436 T486 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16437 T486 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16437 T486 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16438 T486 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16438 T486 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16439 T486 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16439 T486 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16440 T486 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16440 T486 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 16441 T486 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 16453 T486 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 16457 T486 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 16459 T486 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 16461 T486 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty4
[junit4:junit4]   2> 16463 T486 oass.SolrIndexSearcher.<init> Opening Searcher@ff577 main
[junit4:junit4]   2> 16464 T486 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 16464 T486 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 16464 T486 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 16472 T486 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 16524 T487 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@ff577 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 16531 T486 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 16531 T486 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 16893 T421 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16894 T421 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:37523_s",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:37523/s"}
[junit4:junit4]   2> 16894 T421 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 16895 T421 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 16910 T469 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16910 T420 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16910 T426 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16910 T439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16910 T454 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16910 T485 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17534 T486 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 17534 T486 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:37523/s collection:collection1 shard:shard2
[junit4:junit4]   2> 17539 T486 oasc.ZkController.register We are http://127.0.0.1:37523/s/collection1/ and leader is http://127.0.0.1:16649/s/collection1/
[junit4:junit4]   2> 17539 T486 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:37523/s
[junit4:junit4]   2> 17539 T486 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 17539 T486 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C88 name=collection1 org.apache.solr.core.SolrCore@1fd097d9 url=http://127.0.0.1:37523/s/collection1 node=127.0.0.1:37523_s C88_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:37523_s, base_url=http://127.0.0.1:37523/s}
[junit4:junit4]   2> 17540 T488 C88 P37523 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 17540 T486 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17541 T488 C88 P37523 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 17541 T488 C88 P37523 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 17542 T488 C88 P37523 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 17542 T397 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 17542 T397 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 17543 T397 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 17550 T488 C88 P37523 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 17562 T397 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17564 T397 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4]   2> 17565 T397 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C87_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:21808_s, base_url=http://127.0.0.1:21808/s}
[junit4:junit4]   2> 18336 T472 C87 P21808 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:11927/s/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 18346 T472 C87 P21808 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 18364 T472 C87 P21808 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:21808/s START replicas=[http://127.0.0.1:11927/s/collection1/] nUpdates=100
[junit4:junit4]   2> 18375 T472 C87 P21808 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 18376 T472 C87 P21808 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 18376 T472 C87 P21808 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 18376 T472 C87 P21808 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 18376 T472 C87 P21808 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 18376 T472 C87 P21808 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:11927/s/collection1/. core=collection1
[junit4:junit4]   2> 18377 T472 C87 P21808 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 18416 T421 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 18417 T421 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:37523_s",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:37523/s"}
[junit4:junit4]   2> ASYNC  NEW_CORE C89 name=collection1 org.apache.solr.core.SolrCore@22c8af6b url=http://127.0.0.1:11927/s/collection1 node=127.0.0.1:11927_s C89_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:11927_s, base_url=http://127.0.0.1:11927/s, leader=true}
[junit4:junit4]   2> 18431 T430 C89 P11927 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=7 
[junit4:junit4]   2> 18436 T454 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18436 T469 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18436 T439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18436 T420 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18436 T426 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18436 T485 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18449 T431 C89 P11927 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 18450 T431 C89 P11927 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty1
[junit4:junit4]   2> 18454 T431 C89 P11927 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@59ce2c7; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 18455 T431 C89 P11927 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 18459 T431 C89 P11927 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@59ce2c7; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@59ce2c7; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 18460 T431 C89 P11927 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 18461 T431 C89 P11927 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty1
[junit4:junit4]   2> 18461 T431 C89 P11927 oass.SolrIndexSearcher.<init> Opening Searcher@2514265a realtime
[junit4:junit4]   2> 18462 T431 C89 P11927 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 18462 T431 C89 P11927 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 38
[junit4:junit4]   2> 18521 T472 C87 P21808 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 18522 T472 C87 P21808 oascsi.HttpClientUtil.createClient Creating new http client, config:connTimeout=5000&socketTimeout=20000&allowCompression=false&maxConnections=10000&maxConnectionsPerHost=10000
[junit4:junit4]   2> 18548 T472 C87 P21808 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 18568 T397 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 18606 T432 C89 P11927 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 18607 T432 C89 P11927 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=57 
[junit4:junit4]   2> 18608 T472 C87 P21808 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 18609 T472 C87 P21808 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 18609 T472 C87 P21808 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 18612 T432 C89 P11927 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty1
[junit4:junit4]   2> 18613 T432 C89 P11927 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty1/index
[junit4:junit4]   2> 18614 T432 C89 P11927 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=2 
[junit4:junit4]   2> 18615 T472 C87 P21808 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 18616 T472 C87 P21808 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty3/index.20130205144002998 forceNew:false
[junit4:junit4]   2> 18617 T472 C87 P21808 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty3
[junit4:junit4]   2> 18618 T472 C87 P21808 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty3/index.20130205144002998 lockFactory=org.apache.lucene.store.NativeFSLockFactory@19464a92; maxCacheMB=48.0 maxMergeSizeMB=4.0)) fullCopy=true
[junit4:junit4]   2> 18631 T432 C89 P11927 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=4 
[junit4:junit4]   2> 18645 T472 C87 P21808 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty3/index
[junit4:junit4]   2> 18646 T472 C87 P21808 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 18650 T472 C87 P21808 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130205144002998
[junit4:junit4]   2> 18651 T472 C87 P21808 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty3
[junit4:junit4]   2> 18652 T472 C87 P21808 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty3
[junit4:junit4]   2> 18653 T472 C87 P21808 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty3/index/ new=./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty3/index.20130205144002998
[junit4:junit4]   2> 18657 T472 C87 P21808 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty3/index.20130205144002998 lockFactory=org.apache.lucene.store.NativeFSLockFactory@19464a92; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 18658 T472 C87 P21808 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 18660 T472 C87 P21808 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty3
[junit4:junit4]   2> 18665 T472 C87 P21808 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 18665 T472 C87 P21808 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 18665 T472 C87 P21808 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 18667 T472 C87 P21808 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty3/index.20130205144002998
[junit4:junit4]   2> 18668 T472 C87 P21808 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty3
[junit4:junit4]   2> 18671 T472 C87 P21808 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty3/index.20130205144002998 lockFactory=org.apache.lucene.store.NativeFSLockFactory@19464a92; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 18672 T472 C87 P21808 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 18672 T472 C87 P21808 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 18673 T472 C87 P21808 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty3
[junit4:junit4]   2> 18673 T472 C87 P21808 oass.SolrIndexSearcher.<init> Opening Searcher@4142cea6 main
[junit4:junit4]   2> 18675 T471 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4142cea6 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 18676 T471 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty3/index
[junit4:junit4]   2> 18676 T472 C87 P21808 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 18679 T472 C87 P21808 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty3/index.20130205144002998 lockFactory=org.apache.lucene.store.NativeFSLockFactory@19464a92; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty3/index.20130205144002998 lockFactory=org.apache.lucene.store.NativeFSLockFactory@19464a92; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 18680 T472 C87 P21808 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 18681 T472 C87 P21808 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty3
[junit4:junit4]   2> 18681 T472 C87 P21808 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 18682 T472 C87 P21808 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty3/index.20130205144002998
[junit4:junit4]   2> 18682 T472 C87 P21808 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty3/index
[junit4:junit4]   2> 18682 T472 C87 P21808 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 18682 T472 C87 P21808 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 18682 T472 C87 P21808 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 18683 T472 C87 P21808 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 18690 T472 C87 P21808 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 19571 T397 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19944 T421 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19945 T421 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:21808_s",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:21808/s"}
[junit4:junit4]   2> 19965 T485 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19965 T469 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19965 T426 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19965 T439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19965 T420 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19965 T454 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2>  C88_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:37523_s, base_url=http://127.0.0.1:37523/s}
[junit4:junit4]   2> 20568 T488 C88 P37523 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:16649/s/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 20568 T488 C88 P37523 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:37523/s START replicas=[http://127.0.0.1:16649/s/collection1/] nUpdates=100
[junit4:junit4]   2> 20569 T488 C88 P37523 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 20569 T488 C88 P37523 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 20569 T488 C88 P37523 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 20569 T488 C88 P37523 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 20569 T488 C88 P37523 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 20570 T488 C88 P37523 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:16649/s/collection1/. core=collection1
[junit4:junit4]   2> 20571 T488 C88 P37523 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 20574 T397 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> ASYNC  NEW_CORE C90 name=collection1 org.apache.solr.core.SolrCore@40aa18ef url=http://127.0.0.1:16649/s/collection1 node=127.0.0.1:16649_s C90_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:16649_s, base_url=http://127.0.0.1:16649/s, leader=true}
[junit4:junit4]   2> 20581 T445 C90 P16649 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 20589 T446 C90 P16649 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 20590 T446 C90 P16649 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty2
[junit4:junit4]   2> 20594 T446 C90 P16649 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6756223f; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 20595 T446 C90 P16649 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 20599 T446 C90 P16649 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6756223f; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6756223f; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 20599 T446 C90 P16649 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 20600 T446 C90 P16649 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty2
[junit4:junit4]   2> 20601 T446 C90 P16649 oass.SolrIndexSearcher.<init> Opening Searcher@1ecc75f2 realtime
[junit4:junit4]   2> 20601 T446 C90 P16649 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 20602 T446 C90 P16649 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 13
[junit4:junit4]   2> 20603 T488 C88 P37523 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 20603 T488 C88 P37523 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 20606 T447 C90 P16649 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 20606 T447 C90 P16649 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 20607 T488 C88 P37523 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 20608 T488 C88 P37523 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 20608 T488 C88 P37523 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 20611 T447 C90 P16649 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty2
[junit4:junit4]   2> 20612 T447 C90 P16649 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty2/index
[junit4:junit4]   2> 20612 T447 C90 P16649 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=2 
[junit4:junit4]   2> 20613 T488 C88 P37523 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 20614 T488 C88 P37523 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty4/index.20130205144004996 forceNew:false
[junit4:junit4]   2> 20615 T488 C88 P37523 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty4
[junit4:junit4]   2> 20616 T488 C88 P37523 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty4/index.20130205144004996 lockFactory=org.apache.lucene.store.NativeFSLockFactory@ffaf942; maxCacheMB=48.0 maxMergeSizeMB=4.0)) fullCopy=true
[junit4:junit4]   2> 20632 T447 C90 P16649 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> 20634 T488 C88 P37523 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty4/index
[junit4:junit4]   2> 20634 T488 C88 P37523 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 20635 T488 C88 P37523 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130205144004996
[junit4:junit4]   2> 20636 T488 C88 P37523 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty4
[junit4:junit4]   2> 20637 T488 C88 P37523 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty4
[junit4:junit4]   2> 20637 T488 C88 P37523 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty4/index/ new=./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty4/index.20130205144004996
[junit4:junit4]   2> 20641 T488 C88 P37523 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty4/index.20130205144004996 lockFactory=org.apache.lucene.store.NativeFSLockFactory@ffaf942; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 20641 T488 C88 P37523 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 20642 T488 C88 P37523 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty4
[junit4:junit4]   2> 20643 T488 C88 P37523 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 20643 T488 C88 P37523 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 20643 T488 C88 P37523 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 20644 T488 C88 P37523 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty4/index.20130205144004996
[junit4:junit4]   2> 20645 T488 C88 P37523 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty4
[junit4:junit4]   2> 20648 T488 C88 P37523 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty4/index.20130205144004996 lockFactory=org.apache.lucene.store.NativeFSLockFactory@ffaf942; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 20649 T488 C88 P37523 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 20649 T488 C88 P37523 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 20649 T488 C88 P37523 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty4
[junit4:junit4]   2> 20650 T488 C88 P37523 oass.SolrIndexSearcher.<init> Opening Searcher@1efa3571 main
[junit4:junit4]   2> 20652 T487 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1efa3571 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 20652 T487 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty4/index
[junit4:junit4]   2> 20653 T488 C88 P37523 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 20656 T488 C88 P37523 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty4/index.20130205144004996 lockFactory=org.apache.lucene.store.NativeFSLockFactory@ffaf942; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty4/index.20130205144004996 lockFactory=org.apache.lucene.store.NativeFSLockFactory@ffaf942; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 20656 T488 C88 P37523 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 20658 T488 C88 P37523 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty4
[junit4:junit4]   2> 20658 T488 C88 P37523 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 20658 T488 C88 P37523 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty4/index.20130205144004996
[junit4:junit4]   2> 20658 T488 C88 P37523 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty4/index
[junit4:junit4]   2> 20659 T488 C88 P37523 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 20659 T488 C88 P37523 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 20659 T488 C88 P37523 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 20659 T488 C88 P37523 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 20666 T488 C88 P37523 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 21473 T421 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 21474 T421 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:37523_s",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:37523/s"}
[junit4:junit4]   2> 21490 T469 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 21490 T426 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 21490 T485 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 21490 T439 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 21490 T420 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 21490 T454 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 21577 T397 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 21579 T397 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4]   2> ASYNC  NEW_CORE C91 name=collection1 org.apache.solr.core.SolrCore@2c117afd url=http://127.0.0.1:15496/s/collection1 node=127.0.0.1:15496_s C91_STATE=coll:control_collection core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:15496_s, base_url=http://127.0.0.1:15496/s, leader=true}
[junit4:junit4]   2> 21687 T412 C91 P15496 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/control/data
[junit4:junit4]   2> 21692 T412 C91 P15496 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@19bc3c3c; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 21693 T412 C91 P15496 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 21698 T412 C91 P15496 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/control/data
[junit4:junit4]   2> 21699 T412 C91 P15496 oass.SolrIndexSearcher.<init> Opening Searcher@22f4d970 realtime
[junit4:junit4]   2> 21699 T412 C91 P15496 /update {wt=javabin&version=2} {deleteByQuery=*:* (-1426134669531807744)} 0 37
[junit4:junit4]   2> 21723 T448 C90 P16649 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty2
[junit4:junit4]   2> 21747 T433 C89 P11927 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty1
[junit4:junit4]   2>  C88_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:37523_s, base_url=http://127.0.0.1:37523/s}
[junit4:junit4]   2> 21755 T475 C88 P37523 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty4
[junit4:junit4]   2> 21756 T475 C88 P37523 /update {update.distrib=FROMLEADER&_version_=-1426134669569556480&update.from=http://127.0.0.1:16649/s/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1426134669569556480)} 0 3
[junit4:junit4]   2>  C87_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:21808_s, base_url=http://127.0.0.1:21808/s}
[junit4:junit4]   2> 21760 T459 C87 P21808 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360067984389/jetty3
[junit4:junit4]   2> 21761 T459 C87 P21808 /update {update.distrib=FROMLEADER&_version_=-1426134669594722304&update.from=http://127.0.0.1:11927/s/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1426134669594722304)} 0 3
[junit4:junit4]   2> 21762 T433 C89 P11927 /update {update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1426134669594722304)} 0 17
[junit4:junit4]   2> 21763 T448 C90 P16649 /update {wt=javabin&version=2} {deleteByQuery=*:* (-1426134669569556480)} 0 49
[junit4:junit4]   2> 21773 T412 C91 P15496 /update {wt=javabin&version=2} {add=[1 (1426134669618839552)]} 0 5
[junit4:junit4]   2> 21808 T459 C87 P21808 /update {distrib.from=http://127.0.0.1:11927/s/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1426134669646102528)]} 0 5
[junit4:junit4]   2> 21808 T433 C89 P11927 /update {distrib.from=http://127.0.0.1:16649/s/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1426134669646102528)]} 0 13
[junit4:junit4]   2> 21809 T448 C90 P16649 /update {wt=javabin&version=2} {add=[1]} 0 33
[junit4:junit4]   2> 21815 T412 C91 P15496 /update {wt=javabin&version=2} {add=[2 (1426134669664976896)]} 0 3
[junit4:junit4]   2> 21833 T475 C88 P37523 /update {distrib.from=http://127.0.0.1:16649/s/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1426134669671268352)]} 0 7
[junit4:junit4]   2> 21834 T448 C90 P16649 /update {wt=javabin&version=2} {add=[2 (1426134669671268352)]} 0 16
[junit4:junit4]   2> 21841 T412 C91 P15496 /update {wt=javabin&version=2} {add=[3 (1426134669691191296)]} 0 4
[junit4:junit4]   2> 21856 T475 C88 P37523 /update {distrib.from=http://127.0.0.1:16649/s/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[3 (1426134669698531328)]} 0 5
[junit4:junit4]   2> 21857 T448 C90 P16649 /update {wt=javabin&version=2} {add=[3 (1426134669698531328)]} 0 13
[junit4:junit4]   2> 21864 T412 C91 P15496 /update {wt=javabin&version=2} {add=[4 (1426134669715308544)]} 0 4
[junit4:junit4]   2> 21880 T459 C87 P21808 /update {distrib.from=http://127.0.0.1:11927/s/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[4 (1426134669725794304)]} 0 3
[junit4:junit4]   2> 21881 T433 C89 P11927 /update {distrib.from=http://127.0.0.1:16649/s/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[4 (1426134669725794304)]} 0 10
[junit4:junit4]   2> 21882 T448 C90 P16649 /update {wt=javabin&version=2} {add=[4]} 0 15
[junit4:junit4]   2> 21888 T412 C91 P15496 /update {wt=javabin&version=2} {add=[5 (1426134669741522944)]} 0 3
[junit4:junit4]   2> 21908 T475 C88 P37523 /update {distrib.from=http://127.0.0.1:16649/s/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[5 (1426134669750960128)]} 0 5
[junit4:junit4]   2> 21909 T449 C90 P16649 /update {distrib.from=http://127.0.0.1:11927/s/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[5 (1426134669750960128)]} 0 15
[junit4:junit4]   2> 21910 T434 C89 P11927 /update {wt=javabin&version=2} {add=[5]} 0 19
[junit4:junit4]   2> 21916 T412 C91 P15496 /update {wt=javabin&version=2} {add=[6 (1426134669770883072)]} 0 3
[junit4:junit4]   2> 21932 T475 C88 P37523 /update {distrib.from=http://127.0.0.1:16649/s/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[6 (1426134669778223104)]} 0 5
[junit4:junit4]   2> 21933 T448 C90 P16649 /update {wt=javabin&version=2} {add=[6 (1426134669778223104)]} 0 12
[junit4:junit4]   2> 21940 T412 C91 P15496 /update {wt=javabin&version=2} {add=[7 (1426134669795000320)]} 0 3
[junit4:junit4]   2> 21960 T475 C88 P37523 /update {distrib.from=http://127.0.0.1:16649/s/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[7 (1426134669806534656)]} 0 5
[junit4:junit4]   2> 21961 T449 C90 P16649 /update {distrib.from=http://127.0.0.1:11927/s/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[7 (1426134669806534656)]} 0 14
[junit4:junit4]   2> 21962 T434 C89 P11927 /update {wt=javabin&version=2} {add=[7]} 0 19
[junit4:junit4]   2> 21968 T412 C91 P15496 /update {wt=javabin&version=2} {add=[8 (1426134669825409024)]} 0 3
[junit4:junit4]   2> 21986 T459 C87 P21808 /update {distrib.from=http://127.0.0.1:11927/s/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[8 (1426134669835894784)]} 0 4
[junit4:junit4]   2> 21987 T433 C89 P11927 /update {distrib.from=http://127.0.0.1:16649/s/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[8 (1426134669835894784)]} 0 12
[junit4:junit4]   2> 21988 T448 C90 P16649 /update {wt=javabin&version=2} {add=[8]} 0 17
[junit4:junit4]   2> 21995 T412 C91 P15496 /update {wt=javabin&version=2} {add=[9 (1426134669852672000)]} 0 4
[junit4:junit4]   2> 22016 T475 C88 P37523 /update {distrib.from=http://127.0.0.1:16649/s/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[9 (1426134669864206336)]} 0 5
[junit4:junit4]   2> 22017 T449 C90 P16649 /update {distrib.from=http://127.0.0.1:1

[...truncated too long message...]

 Method)
[junit4:junit4]   2> 	        at java.util.TimerThread.mainLoop(Timer.java:552)
[junit4:junit4]   2> 	        at java.util.TimerThread.run(Timer.java:505)
[junit4:junit4]   2> 	  20) Thread[id=474, name=qtp1373514468-474 Acceptor0 SocketConnector@127.0.0.1:37523, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at java.net.PlainSocketImpl.socketAccept(Native Method)
[junit4:junit4]   2> 	        at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
[junit4:junit4]   2> 	        at java.net.ServerSocket.implAccept(ServerSocket.java:522)
[junit4:junit4]   2> 	        at java.net.ServerSocket.accept(ServerSocket.java:490)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.bio.SocketConnector.accept(SocketConnector.java:117)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  21) Thread[id=532, name=searcherExecutor-600-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  22) Thread[id=471, name=searcherExecutor-574-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  23) Thread[id=484, name=TEST-BasicDistributedZkTest.testDistribSearch-seed#[B5C653350651FA91]-SendThread(localhost:40554), state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.nio.ch.KQueueArrayWrapper.interrupt(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.KQueueArrayWrapper.interrupt(KQueueArrayWrapper.java:164)
[junit4:junit4]   2> 	        at sun.nio.ch.KQueueSelectorImpl.wakeup(KQueueSelectorImpl.java:245)
[junit4:junit4]   2> 	        at java.nio.channels.spi.AbstractSelector$1.interrupt(AbstractSelector.java:210)
[junit4:junit4]   2> 	        at java.nio.channels.spi.AbstractSelector.begin(AbstractSelector.java:216)
[junit4:junit4]   2> 	        at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:102)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
[junit4:junit4]   2> 	  24) Thread[id=538, name=searcherExecutor-612-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> NOTE: test params are: codec=Lucene42: {n_f1=PostingsFormat(name=Asserting), foo_b=PostingsFormat(name=MockRandom), foo_d=MockFixedIntBlock(blockSize=688), foo_f=PostingsFormat(name=MockRandom), n_tl1=PostingsFormat(name=Asserting), n_d1=MockVariableIntBlock(baseBlockSize=69), rnd_b=MockFixedIntBlock(blockSize=688), intDefault=MockVariableIntBlock(baseBlockSize=69), n_td1=PostingsFormat(name=Asserting), timestamp=MockVariableIntBlock(baseBlockSize=69), id=MockFixedIntBlock(blockSize=688), range_facet_sl=PostingsFormat(name=MockRandom), range_facet_si=PostingsFormat(name=Asserting), oddField_s=PostingsFormat(name=MockRandom), foo_i=PostingsFormat(name=Asserting), multiDefault=PostingsFormat(name=Asserting), n_tf1=MockVariableIntBlock(baseBlockSize=69), n_dt1=PostingsFormat(name=Asserting), n_ti1=MockFixedIntBlock(blockSize=688), range_facet_l=MockVariableIntBlock(baseBlockSize=69), text=PostingsFormat(name=MockRandom), _version_=MockVariableIntBlock(baseBlockSize=69), val_i=MockFixedIntBlock(blockSize=688), SubjectTerms_mfacet=MockVariableIntBlock(baseBlockSize=69), a_t=MockVariableIntBlock(baseBlockSize=69), n_tdt1=MockVariableIntBlock(baseBlockSize=69), other_tl1=MockVariableIntBlock(baseBlockSize=69), n_l1=MockVariableIntBlock(baseBlockSize=69), a_si=PostingsFormat(name=Asserting)}, sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=et_EE, timezone=Asia/Damascus
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=26,free=153391040,total=233832448
[junit4:junit4]   2> NOTE: All tests run in this JVM: [TestPseudoReturnFields, TestPostingsSolrHighlighter, TestLazyCores, TestGroupingSearch, SolrCoreCheckLockOnStartupTest, JsonLoaderTest, SearchHandlerTest, DirectSolrConnectionTest, TestReload, TestIBSimilarityFactory, TestJmxMonitoredMap, TestIndexSearcher, SuggesterWFSTTest, CurrencyFieldTest, TestQueryTypes, FieldAnalysisRequestHandlerTest, ScriptEngineTest, TestRangeQuery, TestDocumentBuilder, DirectUpdateHandlerOptimizeTest, TestSystemIdResolver, DateMathParserTest, CSVRequestHandlerTest, MBeansHandlerTest, TestSuggestSpellingConverter, ExternalFileFieldSortTest, SuggesterFSTTest, QueryElevationComponentTest, TestCoreContainer, TestLMJelinekMercerSimilarityFactory, TestLMDirichletSimilarityFactory, CopyFieldTest, TestElisionMultitermQuery, TestComponentsName, ChaosMonkeyNothingIsSafeTest, ChaosMonkeySafeLeaderTest, TestBinaryResponseWriter, SimpleFacetsTest, TestFastOutputStream, TestSearchPerf, TestPartialUpdateDeduplication, NotRequiredUniqueKeyTest, ClusterStateUpdateTest, TestFastLRUCache, SpatialFilterTest, AutoCommitTest, JSONWriterTest, TestExtendedDismaxParser, DefaultValueUpdateProcessorTest, FileBasedSpellCheckerTest, DateFieldTest, TestQuerySenderNoQuery, TestSolrQueryParser, ResourceLoaderTest, TestCharFilters, TestLFUCache, PluginInfoTest, TestWordDelimiterFilterFactory, BasicDistributedZkTest]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=BasicDistributedZkTest -Dtests.seed=B5C653350651FA91 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=et_EE -Dtests.timezone=Asia/Damascus -Dtests.file.encoding=ISO-8859-1
[junit4:junit4] ERROR   0.00s J0 | BasicDistributedZkTest (suite) <<<
[junit4:junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([B5C653350651FA91]:0)
[junit4:junit4] Completed on J0 in 7223.57s, 1 test, 2 errors <<< FAILURES!

[...truncated 11 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:346: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/common-build.xml:432: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:1176: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:841: There were test failures: 249 suites, 1038 tests, 1 suite-level error, 1 error, 23 ignored (5 assumptions)

Total time: 144 minutes 50 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-trunk-java7 - Build # 3709 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-java7/3709/

2 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([38B502A79FD73A82]:0)


FAILED:  org.apache.solr.cloud.BasicDistributedZkTest.testDistribSearch

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([38B502A79FD73A82]:0)




Build Log:
[...truncated 9503 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
[junit4:junit4]   2> 0 T820 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /i/
[junit4:junit4]   2> 5 T820 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZkTest-1360035917541
[junit4:junit4]   2> 6 T820 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 7 T821 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 11 T821 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 11 T821 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4]   2> 11 T821 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4]   2> 12 T821 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 13 T821 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4]   2> 108 T820 oasc.ZkTestServer.run start zk server on port:58698
[junit4:junit4]   2> 109 T820 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:58698 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@4a7497ee
[junit4:junit4]   2> 109 T820 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 110 T826 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:58698. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 110 T826 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:58698, initiating session
[junit4:junit4]   2> 111 T822 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:58699
[junit4:junit4]   2> 111 T822 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:58699
[junit4:junit4]   2> 112 T824 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 117 T824 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca8752eed0000 with negotiated timeout 10000 for client /140.211.11.196:58699
[junit4:junit4]   2> 117 T826 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:58698, sessionid = 0x13ca8752eed0000, negotiated timeout = 10000
[junit4:junit4]   2> 117 T827 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4a7497ee name:ZooKeeperConnection Watcher:127.0.0.1:58698 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 118 T820 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 118 T820 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 162 T825 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca8752eed0000
[junit4:junit4]   2> 164 T827 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 164 T822 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:58699 which had sessionid 0x13ca8752eed0000
[junit4:junit4]   2> 164 T820 oaz.ZooKeeper.close Session: 0x13ca8752eed0000 closed
[junit4:junit4]   2> 164 T820 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:58698/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@4c155c72
[junit4:junit4]   2> 165 T820 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 165 T828 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:58698. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 166 T828 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:58698, initiating session
[junit4:junit4]   2> 166 T822 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:58701
[junit4:junit4]   2> 166 T822 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:58701
[junit4:junit4]   2> 178 T824 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca8752eed0001 with negotiated timeout 10000 for client /140.211.11.196:58701
[junit4:junit4]   2> 178 T828 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:58698, sessionid = 0x13ca8752eed0001, negotiated timeout = 10000
[junit4:junit4]   2> 179 T829 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4c155c72 name:ZooKeeperConnection Watcher:127.0.0.1:58698/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 179 T820 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 179 T820 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 187 T820 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 207 T820 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 210 T820 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 221 T820 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 222 T820 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 251 T820 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 252 T820 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 357 T820 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 358 T820 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 362 T820 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 363 T820 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 372 T820 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 373 T820 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 376 T820 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 377 T820 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 381 T820 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 382 T820 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 386 T820 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 387 T820 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 398 T820 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 399 T820 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 403 T825 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca8752eed0001
[junit4:junit4]   2> 419 T829 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 419 T822 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:58701 which had sessionid 0x13ca8752eed0001
[junit4:junit4]   2> 419 T820 oaz.ZooKeeper.close Session: 0x13ca8752eed0001 closed
[junit4:junit4]   2> 657 T820 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 662 T820 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:58702
[junit4:junit4]   2> 663 T820 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 663 T820 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 664 T820 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360035917955
[junit4:junit4]   2> 664 T820 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360035917955/solr.xml
[junit4:junit4]   2> 665 T820 oasc.CoreContainer.<init> New CoreContainer 214051420
[junit4:junit4]   2> 665 T820 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360035917955/'
[junit4:junit4]   2> 666 T820 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360035917955/'
[junit4:junit4]   2> 741 T820 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 741 T820 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 742 T820 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 742 T820 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 743 T820 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 743 T820 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 744 T820 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 744 T820 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 745 T820 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 745 T820 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 773 T820 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 799 T820 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:58698/solr
[junit4:junit4]   2> 799 T820 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 800 T820 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:58698 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@584f50db
[junit4:junit4]   2> 801 T820 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 802 T839 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:58698. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 803 T839 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:58698, initiating session
[junit4:junit4]   2> 803 T822 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:58703
[junit4:junit4]   2> 804 T822 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:58703
[junit4:junit4]   2> 806 T824 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca8752eed0002 with negotiated timeout 20000 for client /140.211.11.196:58703
[junit4:junit4]   2> 806 T839 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:58698, sessionid = 0x13ca8752eed0002, negotiated timeout = 20000
[junit4:junit4]   2> 806 T840 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@584f50db name:ZooKeeperConnection Watcher:127.0.0.1:58698 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 807 T820 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 808 T825 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca8752eed0002
[junit4:junit4]   2> 826 T840 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 826 T822 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:58703 which had sessionid 0x13ca8752eed0002
[junit4:junit4]   2> 826 T820 oaz.ZooKeeper.close Session: 0x13ca8752eed0002 closed
[junit4:junit4]   2> 827 T820 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 841 T820 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:58698/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@4784933a
[junit4:junit4]   2> 842 T820 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 843 T841 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:58698. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 843 T841 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:58698, initiating session
[junit4:junit4]   2> 843 T822 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:58704
[junit4:junit4]   2> 844 T822 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:58704
[junit4:junit4]   2> 851 T824 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca8752eed0003 with negotiated timeout 20000 for client /140.211.11.196:58704
[junit4:junit4]   2> 851 T841 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:58698, sessionid = 0x13ca8752eed0003, negotiated timeout = 20000
[junit4:junit4]   2> 851 T842 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4784933a name:ZooKeeperConnection Watcher:127.0.0.1:58698/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 852 T820 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 896 T825 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca8752eed0003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 898 T820 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 901 T820 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58702_i
[junit4:junit4]   2> 902 T825 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca8752eed0003 type:delete cxid:0x7 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:58702_i Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:58702_i
[junit4:junit4]   2> 928 T820 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:58702_i
[junit4:junit4]   2> 935 T820 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 959 T825 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca8752eed0003 type:delete cxid:0x15 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 975 T820 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 980 T820 oasc.Overseer.start Overseer (id=89131313892491267-127.0.0.1:58702_i-n_0000000000) starting
[junit4:junit4]   2> 980 T825 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca8752eed0003 type:create cxid:0x1a zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 994 T825 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca8752eed0003 type:create cxid:0x1b zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 995 T825 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca8752eed0003 type:create cxid:0x1c zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 1001 T825 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca8752eed0003 type:create cxid:0x1d zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 1024 T844 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 1025 T820 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 1035 T820 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 1037 T843 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 1046 T845 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360035917955/collection1
[junit4:junit4]   2> 1046 T845 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 1047 T845 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 1047 T845 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 1049 T845 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360035917955/collection1/'
[junit4:junit4]   2> 1051 T845 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360035917955/collection1/lib/README' to classloader
[junit4:junit4]   2> 1051 T845 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360035917955/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 1129 T845 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1219 T845 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1320 T845 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 1346 T845 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 2137 T845 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 2150 T845 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 2158 T845 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 2184 T845 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2191 T845 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2196 T845 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2197 T845 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2199 T845 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360035917955/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/control/data/
[junit4:junit4]   2> 2199 T845 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2394c833
[junit4:junit4]   2> 2200 T845 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 2245 T845 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/control/data forceNew:false
[junit4:junit4]   2> 2246 T845 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/control/data
[junit4:junit4]   2> 2247 T845 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/control/data/index/
[junit4:junit4]   2> 2247 T845 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 2249 T845 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/control/data/index forceNew:false
[junit4:junit4]   2> 2274 T845 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@790d9a3a),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 2275 T845 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 2276 T845 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/control/data/index
[junit4:junit4]   2> 2277 T845 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 2279 T845 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 2279 T845 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 2279 T845 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 2280 T845 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 2281 T845 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 2281 T845 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 2281 T845 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 2282 T845 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 2282 T845 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 2283 T845 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 2283 T845 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 2284 T845 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 2284 T845 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 2285 T845 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 2285 T845 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 2286 T845 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 2286 T845 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2287 T845 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2287 T845 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2288 T845 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2288 T845 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2289 T845 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2289 T845 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2290 T845 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2290 T845 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 2291 T845 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 2329 T845 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 2337 T845 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 2339 T845 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 2343 T845 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/control/data
[junit4:junit4]   2> 2346 T845 oass.SolrIndexSearcher.<init> Opening Searcher@2b2aa4d3 main
[junit4:junit4]   2> 2347 T845 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 2347 T845 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 2348 T845 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 2490 T845 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 2620 T846 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2b2aa4d3 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 2626 T845 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 2626 T845 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 2627 T825 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca8752eed0003 type:create cxid:0x53 zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 4045 T843 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 4046 T843 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:58702_i",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:58702/i"}
[junit4:junit4]   2> 4046 T843 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4]   2> 4053 T843 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 4054 T825 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca8752eed0003 type:create cxid:0x5b zxid:0x2c txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 4097 T842 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 4666 T845 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 4666 T845 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:58702/i collection:control_collection shard:shard1
[junit4:junit4]   2> 4667 T845 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4]   2> 4704 T825 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca8752eed0003 type:delete cxid:0x75 zxid:0x36 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4]   2> 4705 T845 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 4706 T825 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca8752eed0003 type:create cxid:0x76 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4729 T845 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 4729 T845 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 4730 T845 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:58702/i/collection1/
[junit4:junit4]   2> 4730 T845 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 4730 T845 oasc.SyncStrategy.syncToMe http://127.0.0.1:58702/i/collection1/ has no replicas
[junit4:junit4]   2> 4731 T845 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:58702/i/collection1/
[junit4:junit4]   2> 4731 T845 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4]   2> 4745 T825 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca8752eed0003 type:create cxid:0x80 zxid:0x3b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5604 T843 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 5637 T842 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 5677 T845 oasc.ZkController.register We are http://127.0.0.1:58702/i/collection1/ and leader is http://127.0.0.1:58702/i/collection1/
[junit4:junit4]   2> 5677 T845 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:58702/i
[junit4:junit4]   2> 5677 T845 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 5678 T845 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 5678 T845 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 5681 T845 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 5682 T820 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 5683 T820 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 5684 T820 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 5704 T820 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 5731 T820 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:58698/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@330b6e9
[junit4:junit4]   2> 5732 T820 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 5732 T847 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:58698. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 5733 T847 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:58698, initiating session
[junit4:junit4]   2> 5733 T822 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:58720
[junit4:junit4]   2> 5734 T822 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:58720
[junit4:junit4]   2> 5736 T824 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca8752eed0004 with negotiated timeout 10000 for client /140.211.11.196:58720
[junit4:junit4]   2> 5736 T847 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:58698, sessionid = 0x13ca8752eed0004, negotiated timeout = 10000
[junit4:junit4]   2> 5736 T848 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@330b6e9 name:ZooKeeperConnection Watcher:127.0.0.1:58698/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 5737 T820 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 5738 T820 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 5743 T820 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 5959 T820 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 5962 T820 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:58721
[junit4:junit4]   2> 5963 T820 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 5963 T820 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 5964 T820 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360035923279
[junit4:junit4]   2> 5964 T820 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360035923279/solr.xml
[junit4:junit4]   2> 5965 T820 oasc.CoreContainer.<init> New CoreContainer 1491351555
[junit4:junit4]   2> 5965 T820 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360035923279/'
[junit4:junit4]   2> 5966 T820 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360035923279/'
[junit4:junit4]   2> 6032 T820 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 6032 T820 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 6033 T820 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 6033 T820 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 6034 T820 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 6034 T820 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 6035 T820 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 6035 T820 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 6036 T820 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 6036 T820 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 6063 T820 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 6083 T820 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:58698/solr
[junit4:junit4]   2> 6084 T820 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 6085 T820 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:58698 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@7d03a9b0
[junit4:junit4]   2> 6086 T820 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 6086 T858 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:58698. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 6087 T858 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:58698, initiating session
[junit4:junit4]   2> 6088 T822 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:58722
[junit4:junit4]   2> 6088 T822 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:58722
[junit4:junit4]   2> 6090 T824 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca8752eed0005 with negotiated timeout 20000 for client /140.211.11.196:58722
[junit4:junit4]   2> 6090 T858 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:58698, sessionid = 0x13ca8752eed0005, negotiated timeout = 20000
[junit4:junit4]   2> 6091 T859 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7d03a9b0 name:ZooKeeperConnection Watcher:127.0.0.1:58698 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 6091 T820 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 6093 T825 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca8752eed0005
[junit4:junit4]   2> 6094 T859 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 6095 T822 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:58722 which had sessionid 0x13ca8752eed0005
[junit4:junit4]   2> 6094 T820 oaz.ZooKeeper.close Session: 0x13ca8752eed0005 closed
[junit4:junit4]   2> 6095 T820 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 6109 T820 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:58698/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@29a1387d
[junit4:junit4]   2> 6110 T820 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 6111 T860 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:58698. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 6111 T860 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:58698, initiating session
[junit4:junit4]   2> 6112 T822 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:58723
[junit4:junit4]   2> 6112 T822 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:58723
[junit4:junit4]   2> 6114 T824 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca8752eed0006 with negotiated timeout 20000 for client /140.211.11.196:58723
[junit4:junit4]   2> 6114 T860 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:58698, sessionid = 0x13ca8752eed0006, negotiated timeout = 20000
[junit4:junit4]   2> 6115 T861 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@29a1387d name:ZooKeeperConnection Watcher:127.0.0.1:58698/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 6115 T820 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 6116 T825 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca8752eed0006 type:create cxid:0x1 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6118 T825 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca8752eed0006 type:create cxid:0x2 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6136 T820 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58721_i
[junit4:junit4]   2> 6137 T825 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca8752eed0006 type:delete cxid:0x4 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:58721_i Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:58721_i
[junit4:junit4]   2> 6139 T820 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:58721_i
[junit4:junit4]   2> 6142 T842 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 6143 T848 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 6144 T842 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 6147 T820 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 6155 T862 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360035923279/collection1
[junit4:junit4]   2> 6155 T862 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 6156 T862 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 6156 T862 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 6157 T862 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360035923279/collection1/'
[junit4:junit4]   2> 6159 T862 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360035923279/collection1/lib/README' to classloader
[junit4:junit4]   2> 6159 T862 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360035923279/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 6229 T862 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 6314 T862 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 6415 T862 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 6440 T862 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 7063 T862 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 7073 T862 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 7077 T862 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 7095 T862 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 7100 T862 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 7105 T862 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 7106 T862 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 7108 T862 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360035923279/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty1/
[junit4:junit4]   2> 7108 T862 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2394c833
[junit4:junit4]   2> 7109 T862 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 7110 T862 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty1 forceNew:false
[junit4:junit4]   2> 7110 T862 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty1
[junit4:junit4]   2> 7111 T862 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty1/index/
[junit4:junit4]   2> 7111 T862 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 7113 T862 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty1/index forceNew:false
[junit4:junit4]   2> 7120 T862 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5aa4e1af),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 7120 T862 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 7121 T862 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty1/index
[junit4:junit4]   2> 7122 T862 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 7124 T862 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 7124 T862 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 7124 T862 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 7125 T862 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 7125 T862 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 7126 T862 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 7126 T862 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 7127 T862 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 7127 T862 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 7128 T862 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 7128 T862 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 7129 T862 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 7129 T862 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 7130 T862 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 7130 T862 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 7131 T862 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 7131 T862 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7132 T862 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7132 T862 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7133 T862 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7133 T862 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7134 T862 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7134 T862 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7135 T862 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7135 T862 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 7136 T862 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 7145 T843 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 7146 T843 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:58702_i",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:58702/i"}
[junit4:junit4]   2> 7150 T862 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 7154 T862 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 7162 T862 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 7163 T861 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7163 T842 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7163 T848 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7165 T862 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty1
[junit4:junit4]   2> 7167 T862 oass.SolrIndexSearcher.<init> Opening Searcher@188bbd6b main
[junit4:junit4]   2> 7168 T862 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 7168 T862 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 7169 T862 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 7183 T862 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 7274 T863 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@188bbd6b main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 7342 T862 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 7342 T862 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 8677 T843 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8678 T843 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:58721_i",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:58721/i"}
[junit4:junit4]   2> 8678 T843 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4]   2> 8678 T843 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 8716 T848 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8716 T842 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8716 T861 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 9365 T862 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 9365 T862 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:58721/i collection:collection1 shard:shard1
[junit4:junit4]   2> 9366 T862 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 9397 T825 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca8752eed0006 type:delete cxid:0x45 zxid:0x5b txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 9406 T862 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 9407 T825 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca8752eed0006 type:create cxid:0x46 zxid:0x5c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9410 T862 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 9410 T862 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 9410 T862 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:58721/i/collection1/
[junit4:junit4]   2> 9411 T862 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 9411 T862 oasc.SyncStrategy.syncToMe http://127.0.0.1:58721/i/collection1/ has no replicas
[junit4:junit4]   2> 9411 T862 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:58721/i/collection1/
[junit4:junit4]   2> 9411 T862 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 9428 T825 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca8752eed0006 type:create cxid:0x50 zxid:0x60 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 10223 T843 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 10250 T848 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 10250 T861 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 10250 T842 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 10263 T862 oasc.ZkController.register We are http://127.0.0.1:58721/i/collection1/ and leader is http://127.0.0.1:58721/i/collection1/
[junit4:junit4]   2> 10263 T862 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:58721/i
[junit4:junit4]   2> 10263 T862 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 10263 T862 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 10264 T862 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 10274 T862 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 10276 T820 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 10276 T820 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 10277 T820 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 10474 T820 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 10477 T820 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:58724
[junit4:junit4]   2> 10478 T820 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 10478 T820 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 10479 T820 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360035927830
[junit4:junit4]   2> 10479 T820 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360035927830/solr.xml
[junit4:junit4]   2> 10479 T820 oasc.CoreContainer.<init> New CoreContainer 1052745438
[junit4:junit4]   2> 10480 T820 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360035927830/'
[junit4:junit4]   2> 10480 T820 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360035927830/'
[junit4:junit4]   2> 10535 T820 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 10536 T820 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 10536 T820 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 10537 T820 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 10537 T820 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 10538 T820 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 10538 T820 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 10538 T820 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 10539 T820 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 10539 T820 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 10560 T820 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 10578 T820 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:58698/solr
[junit4:junit4]   2> 10579 T820 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 10580 T820 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:58698 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@351393e4
[junit4:junit4]   2> 10581 T820 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 10581 T873 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:58698. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 10582 T873 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:58698, initiating session
[junit4:junit4]   2> 10582 T822 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:58725
[junit4:junit4]   2> 10583 T822 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:58725
[junit4:junit4]   2> 10589 T824 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca8752eed0007 with negotiated timeout 20000 for client /140.211.11.196:58725
[junit4:junit4]   2> 10589 T873 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:58698, sessionid = 0x13ca8752eed0007, negotiated timeout = 20000
[junit4:junit4]   2> 10589 T874 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@351393e4 name:ZooKeeperConnection Watcher:127.0.0.1:58698 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 10589 T820 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 10591 T825 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca8752eed0007
[junit4:junit4]   2> 10600 T874 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 10600 T822 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:58725 which had sessionid 0x13ca8752eed0007
[junit4:junit4]   2> 10600 T820 oaz.ZooKeeper.close Session: 0x13ca8752eed0007 closed
[junit4:junit4]   2> 10601 T820 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 10612 T820 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:58698/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@627ba26f
[junit4:junit4]   2> 10613 T820 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 10614 T875 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:58698. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 10614 T875 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:58698, initiating session
[junit4:junit4]   2> 10614 T822 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:58726
[junit4:junit4]   2> 10615 T822 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:58726
[junit4:junit4]   2> 10619 T824 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca8752eed0008 with negotiated timeout 20000 for client /140.211.11.196:58726
[junit4:junit4]   2> 10619 T875 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:58698, sessionid = 0x13ca8752eed0008, negotiated timeout = 20000
[junit4:junit4]   2> 10620 T876 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@627ba26f name:ZooKeeperConnection Watcher:127.0.0.1:58698/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 10620 T820 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 10621 T825 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca8752eed0008 type:create cxid:0x1 zxid:0x6d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 10623 T825 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca8752eed0008 type:create cxid:0x2 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 10630 T820 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58724_i
[junit4:junit4]   2> 10631 T825 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca8752eed0008 type:delete cxid:0x4 zxid:0x6f txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:58724_i Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:58724_i
[junit4:junit4]   2> 10633 T820 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:58724_i
[junit4:junit4]   2> 10636 T842 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 10636 T861 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 10636 T848 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 10636 T861 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10636 T848 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10638 T842 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 10653 T820 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 10661 T877 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360035927830/collection1
[junit4:junit4]   2> 10661 T877 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 10662 T877 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 10662 T877 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 10664 T877 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360035927830/collection1/'
[junit4:junit4]   2> 10665 T877 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360035927830/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 10666 T877 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360035927830/collection1/lib/README' to classloader
[junit4:junit4]   2> 10725 T877 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 10803 T877 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 10904 T877 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 10911 T877 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 11801 T843 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11803 T843 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:58721_i",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:58721/i"}
[junit4:junit4]   2> 11824 T848 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11824 T842 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11825 T876 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11832 T861 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11938 T877 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 11966 T877 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 11971 T877 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 12016 T877 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 12023 T877 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 12033 T877 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 12035 T877 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 12038 T877 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360035927830/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty2/
[junit4:junit4]   2> 12038 T877 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2394c833
[junit4:junit4]   2> 12040 T877 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 12041 T877 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty2 forceNew:false
[junit4:junit4]   2> 12042 T877 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty2
[junit4:junit4]   2> 12042 T877 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty2/index/
[junit4:junit4]   2> 12043 T877 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 12044 T877 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty2/index forceNew:false
[junit4:junit4]   2> 12088 T877 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4d130642),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 12089 T877 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 12090 T877 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty2/index
[junit4:junit4]   2> 12091 T877 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 12094 T877 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 12095 T877 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 12095 T877 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 12095 T877 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 12096 T877 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 12096 T877 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 12097 T877 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 12098 T877 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 12098 T877 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 12099 T877 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 12099 T877 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 12100 T877 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 12101 T877 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 12101 T877 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 12101 T877 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 12102 T877 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 12103 T877 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12104 T877 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12105 T877 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12106 T877 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12107 T877 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12107 T877 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12108 T877 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12109 T877 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12110 T877 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 12111 T877 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 12145 T877 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 12152 T877 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 12155 T877 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 12169 T877 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty2
[junit4:junit4]   2> 12173 T877 oass.SolrIndexSearcher.<init> Opening Searcher@7afb6f54 main
[junit4:junit4]   2> 12175 T877 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 12175 T877 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 12176 T877 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 12211 T877 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 12754 T878 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7afb6f54 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 12771 T877 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 12771 T877 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 13354 T843 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 13355 T843 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:58724_i",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:58724/i"}
[junit4:junit4]   2> 13356 T843 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 13356 T843 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 13400 T876 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13400 T842 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13400 T848 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13400 T861 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13787 T877 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 13787 T877 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:58724/i collection:collection1 shard:shard2
[junit4:junit4]   2> 13791 T877 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4]   2> 13821 T825 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca8752eed0008 type:delete cxid:0x44 zxid:0x7e txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
[junit4:junit4]   2> 13826 T877 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 13828 T825 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca8752eed0008 type:create cxid:0x45 zxid:0x7f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 13841 T877 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 13842 T877 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 13842 T877 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:58724/i/collection1/
[junit4:junit4]   2> 13842 T877 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 13843 T877 oasc.SyncStrategy.syncToMe http://127.0.0.1:58724/i/collection1/ has no replicas
[junit4:junit4]   2> 13843 T877 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:58724/i/collection1/
[junit4:junit4]   2> 13843 T877 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4]   2> 13859 T825 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca8752eed0008 type:create cxid:0x4e zxid:0x82 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 14912 T843 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14948 T848 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 14948 T842 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 14948 T861 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 14948 T876 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 15005 T877 oasc.ZkController.register We are http://127.0.0.1:58724/i/collection1/ and leader is http://127.0.0.1:58724/i/collection1/
[junit4:junit4]   2> 15005 T877 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:58724/i
[junit4:junit4]   2> 15005 T877 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 15006 T877 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 15006 T877 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 15009 T877 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 15010 T820 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 15011 T820 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 15011 T820 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 15223 T820 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 15227 T820 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:58727
[junit4:junit4]   2> 15227 T820 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 15228 T820 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 15228 T820 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360035932562
[junit4:junit4]   2> 15229 T820 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360035932562/solr.xml
[junit4:junit4]   2> 15229 T820 oasc.CoreContainer.<init> New CoreContainer 43236172
[junit4:junit4]   2> 15230 T820 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360035932562/'
[junit4:junit4]   2> 15230 T820 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360035932562/'
[junit4:junit4]   2> 15293 T820 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 15294 T820 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 15294 T820 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 15295 T820 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 15295 T820 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 15296 T820 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 15296 T820 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 15297 T820 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 15297 T820 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 15298 T820 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 15325 T820 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 15347 T820 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:58698/solr
[junit4:junit4]   2> 15348 T820 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 15348 T820 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:58698 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@d32d034
[junit4:junit4]   2> 15350 T820 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 15350 T888 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:58698. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 15351 T888 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:58698, initiating session
[junit4:junit4]   2> 15351 T822 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:58728
[junit4:junit4]   2> 15352 T822 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:58728
[junit4:junit4]   2> 15361 T824 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca8752eed0009 with negotiated timeout 20000 for client /140.211.11.196:58728
[junit4:junit4]   2> 15361 T888 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:58698, sessionid = 0x13ca8752eed0009, negotiated timeout = 20000
[junit4:junit4]   2> 15361 T889 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d32d034 name:ZooKeeperConnection Watcher:127.0.0.1:58698 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 15362 T820 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 15363 T825 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca8752eed0009
[junit4:junit4]   2> 15435 T889 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 15435 T822 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:58728 which had sessionid 0x13ca8752eed0009
[junit4:junit4]   2> 15435 T820 oaz.ZooKeeper.close Session: 0x13ca8752eed0009 closed
[junit4:junit4]   2> 15436 T820 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 15451 T820 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:58698/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@7be7f8ea
[junit4:junit4]   2> 15452 T820 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 15452 T890 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:58698. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 15453 T890 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:58698, initiating session
[junit4:junit4]   2> 15453 T822 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:58729
[junit4:junit4]   2> 15454 T822 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:58729
[junit4:junit4]   2> 15457 T824 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca8752eed000a with negotiated timeout 20000 for client /140.211.11.196:58729
[junit4:junit4]   2> 15458 T890 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:58698, sessionid = 0x13ca8752eed000a, negotiated timeout = 20000
[junit4:junit4]   2> 15458 T891 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7be7f8ea name:ZooKeeperConnection Watcher:127.0.0.1:58698/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 15458 T820 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 15460 T825 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca8752eed000a type:create cxid:0x1 zxid:0x8f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 15475 T825 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca8752eed000a type:create cxid:0x2 zxid:0x90 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 15477 T820 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58727_i
[junit4:junit4]   2> 15478 T825 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca8752eed000a type:delete cxid:0x4 zxid:0x91 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:58727_i Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:58727_i
[junit4:junit4]   2> 15491 T820 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:58727_i
[junit4:junit4]   2> 15500 T876 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 15500 T842 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 15501 T848 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 15501 T861 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 15501 T848 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15501 T861 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15502 T876 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 15502 T842 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 15506 T820 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 15515 T892 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360035932562/collection1
[junit4:junit4]   2> 15515 T892 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 15516 T892 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 15516 T892 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 15518 T892 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360035932562/collection1/'
[junit4:junit4]   2> 15519 T892 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360035932562/collection1/lib/README' to classloader
[junit4:junit4]   2> 15519 T892 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360035932562/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 15576 T892 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 15650 T892 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 15751 T892 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 15758 T892 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 16243 T892 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 16252 T892 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 16255 T892 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 16268 T892 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 16272 T892 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 16275 T892 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 16276 T892 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 16277 T892 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360035932562/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty3/
[junit4:junit4]   2> 16277 T892 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2394c833
[junit4:junit4]   2> 16278 T892 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 16278 T892 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty3 forceNew:false
[junit4:junit4]   2> 16279 T892 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty3
[junit4:junit4]   2> 16279 T892 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty3/index/
[junit4:junit4]   2> 16279 T892 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 16280 T892 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty3/index forceNew:false
[junit4:junit4]   2> 16293 T892 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@679fa099),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 16293 T892 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 16294 T892 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty3/index
[junit4:junit4]   2> 16294 T892 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 16296 T892 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 16296 T892 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 16296 T892 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 16297 T892 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 16297 T892 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 16297 T892 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 16298 T892 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 16298 T892 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 16298 T892 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 16299 T892 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 16299 T892 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 16300 T892 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 16300 T892 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 16300 T892 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 16300 T892 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 16301 T892 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 16301 T892 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16302 T892 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16302 T892 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16302 T892 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16303 T892 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16303 T892 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16304 T892 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16304 T892 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16304 T892 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 16305 T892 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 16313 T892 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 16316 T892 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 16318 T892 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 16319 T892 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty3
[junit4:junit4]   2> 16321 T892 oass.SolrIndexSearcher.<init> Opening Searcher@71e9d532 main
[junit4:junit4]   2> 16321 T892 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 16322 T892 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 16322 T892 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 16328 T892 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 16386 T893 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@71e9d532 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 16389 T892 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 16390 T892 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 16474 T843 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16475 T843 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:58724_i",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:58724/i"}
[junit4:junit4]   2> 16505 T843 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:58727_i",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:58727/i"}
[junit4:junit4]   2> 16506 T843 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 16506 T843 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 16532 T848 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16532 T891 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16532 T861 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16532 T876 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16532 T842 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 17392 T892 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 17392 T892 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:58727/i collection:collection1 shard:shard1
[junit4:junit4]   2> 17426 T892 oasc.ZkController.register We are http://127.0.0.1:58727/i/collection1/ and leader is http://127.0.0.1:58721/i/collection1/
[junit4:junit4]   2> 17426 T892 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:58727/i
[junit4:junit4]   2> 17426 T892 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 17427 T892 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C92 name=collection1 org.apache.solr.core.SolrCore@23c83d6f url=http://127.0.0.1:58727/i/collection1 node=127.0.0.1:58727_i C92_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:58727_i, base_url=http://127.0.0.1:58727/i}
[junit4:junit4]   2> 17435 T894 C92 P58727 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 17436 T892 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17437 T894 C92 P58727 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 17437 T894 C92 P58727 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 17437 T894 C92 P58727 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 17438 T820 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 17438 T820 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 17439 T820 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 17440 T894 C92 P58727 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 17628 T820 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 17631 T820 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:58735
[junit4:junit4]   2> 17632 T820 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 17632 T820 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 17633 T820 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360035934996
[junit4:junit4]   2> 17633 T820 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360035934996/solr.xml
[junit4:junit4]   2> 17634 T820 oasc.CoreContainer.<init> New CoreContainer 1006411657
[junit4:junit4]   2> 17634 T820 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360035934996/'
[junit4:junit4]   2> 17635 T820 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360035934996/'
[junit4:junit4]   2> 17691 T820 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 17692 T820 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 17692 T820 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 17693 T820 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 17693 T820 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 17694 T820 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 17694 T820 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 17694 T820 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 17695 T820 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 17695 T820 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 17717 T820 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 17736 T820 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:58698/solr
[junit4:junit4]   2> 17736 T820 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 17737 T820 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:58698 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@2b9a98f1
[junit4:junit4]   2> 17738 T820 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 17739 T904 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:58698. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 17739 T904 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:58698, initiating session
[junit4:junit4]   2> 17740 T822 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:58737
[junit4:junit4]   2> 17740 T822 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:58737
[junit4:junit4]   2> 17749 T824 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca8752eed000b with negotiated timeout 20000 for client /140.211.11.196:58737
[junit4:junit4]   2> 17749 T904 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:58698, sessionid = 0x13ca8752eed000b, negotiated timeout = 20000
[junit4:junit4]   2> 17749 T905 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2b9a98f1 name:ZooKeeperConnection Watcher:127.0.0.1:58698 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 17749 T820 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 17751 T825 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca8752eed000b
[junit4:junit4]   2> 17790 T822 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:58737 which had sessionid 0x13ca8752eed000b
[junit4:junit4]   2> 17790 T905 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 17790 T820 oaz.ZooKeeper.close Session: 0x13ca8752eed000b closed
[junit4:junit4]   2> 17791 T820 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 17806 T820 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:58698/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@71d20652
[junit4:junit4]   2> 17807 T820 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 17807 T906 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:58698. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 17808 T906 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:58698, initiating session
[junit4:junit4]   2> 17808 T822 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:58738
[junit4:junit4]   2> 17809 T822 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:58738
[junit4:junit4]   2> 17838 T824 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca8752eed000c with negotiated timeout 20000 for client /140.211.11.196:58738
[junit4:junit4]   2> 17838 T906 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:58698, sessionid = 0x13ca8752eed000c, negotiated timeout = 20000
[junit4:junit4]   2> 17839 T907 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@71d20652 name:ZooKeeperConnection Watcher:127.0.0.1:58698/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 17839 T820 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 17840 T825 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca8752eed000c type:create cxid:0x1 zxid:0xa1 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 17863 T825 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca8752eed000c type:create cxid:0x2 zxid:0xa2 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 17865 T820 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58735_i
[junit4:junit4]   2> 17866 T825 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca8752eed000c type:delete cxid:0x4 zxid:0xa3 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:58735_i Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:58735_i
[junit4:junit4]   2> 17887 T820 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:58735_i
[junit4:junit4]   2> 17891 T876 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 17891 T842 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 17892 T861 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 17892 T848 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 17892 T891 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 17892 T848 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17892 T861 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17892 T891 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17894 T876 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 17894 T842 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 17908 T820 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 17915 T908 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360035934996/collection1
[junit4:junit4]   2> 17916 T908 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 17916 T908 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 17917 T908 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 17918 T908 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360035934996/collection1/'
[junit4:junit4]   2> 17919 T908 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360035934996/collection1/lib/README' to classloader
[junit4:junit4]   2> 17920 T908 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360035934996/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 17978 T908 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 18050 T908 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 18052 T843 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 18053 T843 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:58727_i",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:58727/i"}
[junit4:junit4]   2> 18074 T861 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18074 T876 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18074 T891 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18074 T842 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18074 T848 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18151 T908 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 18151 T907 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18158 T908 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 18777 T908 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 18788 T908 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 18792 T908 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 18815 T908 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 18820 T908 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 18824 T908 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 18826 T908 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 18827 T908 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360035934996/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty4/
[junit4:junit4]   2> 18827 T908 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2394c833
[junit4:junit4]   2> 18828 T908 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 18829 T908 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty4 forceNew:false
[junit4:junit4]   2> 18830 T908 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty4
[junit4:junit4]   2> 18830 T908 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty4/index/
[junit4:junit4]   2> 18830 T908 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 18831 T908 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty4/index forceNew:false
[junit4:junit4]   2> 18876 T908 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2f21b19b),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 18876 T908 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 18877 T908 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty4/index
[junit4:junit4]   2> 18877 T908 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 18880 T908 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 18880 T908 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 18880 T908 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 18881 T908 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 18881 T908 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 18882 T908 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 18882 T908 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 18883 T908 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 18883 T908 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 18884 T908 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 18884 T908 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 18885 T908 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 18885 T908 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 18886 T908 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 18886 T908 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 18886 T908 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 18887 T908 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 18888 T908 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 18888 T908 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 18889 T908 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 18889 T908 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 18890 T908 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 18890 T908 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 18891 T908 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 18891 T908 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 18892 T908 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 18904 T908 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 18908 T908 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 18910 T908 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 18912 T908 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty4
[junit4:junit4]   2> 18914 T908 oass.SolrIndexSearcher.<init> Opening Searcher@730f58f main
[junit4:junit4]   2> 18915 T908 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 18915 T908 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 18915 T908 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 18945 T908 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 19069 T909 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@730f58f main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 19074 T908 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 19074 T908 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 19580 T843 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19581 T843 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:58735_i",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:58735/i"}
[junit4:junit4]   2> 19581 T843 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 19581 T843 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 19603 T876 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19604 T848 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19603 T907 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19604 T842 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19604 T861 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19604 T891 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20077 T908 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 20077 T908 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:58735/i collection:collection1 shard:shard2
[junit4:junit4]   2> 20081 T908 oasc.ZkController.register We are http://127.0.0.1:58735/i/collection1/ and leader is http://127.0.0.1:58724/i/collection1/
[junit4:junit4]   2> 20082 T908 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:58735/i
[junit4:junit4]   2> 20082 T908 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 20082 T908 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C93 name=collection1 org.apache.solr.core.SolrCore@700133da url=http://127.0.0.1:58735/i/collection1 node=127.0.0.1:58735_i C93_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:58735_i, base_url=http://127.0.0.1:58735/i}
[junit4:junit4]   2> 20083 T910 C93 P58735 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 20083 T908 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 20084 T910 C93 P58735 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 20084 T910 C93 P58735 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 20084 T910 C93 P58735 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 20085 T820 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 20085 T820 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 20086 T910 C93 P58735 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 20086 T820 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 20107 T820 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 20109 T820 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4]   2> 20110 T820 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C92_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:58727_i, base_url=http://127.0.0.1:58727/i}
[junit4:junit4]   2> 20476 T894 C92 P58727 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:58721/i/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 20487 T894 C92 P58727 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 20510 T894 C92 P58727 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:58727/i START replicas=[http://127.0.0.1:58721/i/collection1/] nUpdates=100
[junit4:junit4]   2> 20510 T894 C92 P58727 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 20511 T894 C92 P58727 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 20511 T894 C92 P58727 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 20512 T894 C92 P58727 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 20512 T894 C92 P58727 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 20512 T894 C92 P58727 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:58721/i/collection1/. core=collection1
[junit4:junit4]   2> 20513 T894 C92 P58727 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C94 name=collection1 org.apache.solr.core.SolrCore@6bedc615 url=http://127.0.0.1:58721/i/collection1 node=127.0.0.1:58721_i C94_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:58721_i, base_url=http://127.0.0.1:58721/i, leader=true}
[junit4:junit4]   2> 20523 T852 C94 P58721 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 20552 T853 C94 P58721 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 20552 T853 C94 P58721 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty1
[junit4:junit4]   2> 20557 T853 C94 P58721 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5aa4e1af),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 20557 T853 C94 P58721 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 20566 T853 C94 P58721 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5aa4e1af),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5aa4e1af),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 20566 T853 C94 P58721 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 20568 T853 C94 P58721 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty1
[junit4:junit4]   2> 20568 T853 C94 P58721 oass.SolrIndexSearcher.<init> Opening Searcher@19e269a3 realtime
[junit4:junit4]   2> 20568 T853 C94 P58721 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 20569 T853 C94 P58721 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 35
[junit4:junit4]   2> 20570 T894 C92 P58727 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 20571 T894 C92 P58727 oascsi.HttpClientUtil.createClient Creating new http client, config:connTimeout=5000&socketTimeout=20000&allowCompression=false&maxConnections=10000&maxConnectionsPerHost=10000
[junit4:junit4]   2> 20600 T894 C92 P58727 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 20603 T854 C94 P58721 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 20604 T854 C94 P58721 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 20605 T894 C92 P58727 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 20605 T894 C92 P58727 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 20606 T894 C92 P58727 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 20609 T854 C94 P58721 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty1
[junit4:junit4]   2> 20609 T854 C94 P58721 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty1/index
[junit4:junit4]   2> 20610 T854 C94 P58721 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=2 
[junit4:junit4]   2> 20611 T894 C92 P58727 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 20612 T894 C92 P58727 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty3/index.20130204214538146 forceNew:false
[junit4:junit4]   2> 20613 T894 C92 P58727 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty3
[junit4:junit4]   2> 20614 T894 C92 P58727 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty3/index.20130204214538146 lockFactory=org.apache.lucene.store.NativeFSLockFactory@408f1117) fullCopy=true
[junit4:junit4]   2> 20628 T854 C94 P58721 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=5 
[junit4:junit4]   2> 20635 T894 C92 P58727 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty3/index
[junit4:junit4]   2> 20635 T894 C92 P58727 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 20636 T894 C92 P58727 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130204214538146
[junit4:junit4]   2> 20638 T894 C92 P58727 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty3
[junit4:junit4]   2> 20639 T894 C92 P58727 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty3
[junit4:junit4]   2> 20640 T894 C92 P58727 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty3/index/ new=./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty3/index.20130204214538146
[junit4:junit4]   2> 20643 T894 C92 P58727 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty3/index.20130204214538146 lockFactory=org.apache.lucene.store.NativeFSLockFactory@408f1117),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 20644 T894 C92 P58727 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 20646 T894 C92 P58727 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty3
[junit4:junit4]   2> 20650 T894 C92 P58727 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 20651 T894 C92 P58727 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 20651 T894 C92 P58727 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 20652 T894 C92 P58727 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty3/index.20130204214538146
[junit4:junit4]   2> 20653 T894 C92 P58727 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty3
[junit4:junit4]   2> 20656 T894 C92 P58727 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty3/index.20130204214538146 lockFactory=org.apache.lucene.store.NativeFSLockFactory@408f1117),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 20657 T894 C92 P58727 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 20657 T894 C92 P58727 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 20658 T894 C92 P58727 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty3
[junit4:junit4]   2> 20658 T894 C92 P58727 oass.SolrIndexSearcher.<init> Opening Searcher@36461db6 main
[junit4:junit4]   2> 20661 T893 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@36461db6 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 20661 T893 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty3/index
[junit4:junit4]   2> 20662 T894 C92 P58727 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 20664 T894 C92 P58727 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty3/index.20130204214538146 lockFactory=org.apache.lucene.store.NativeFSLockFactory@408f1117),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty3/index.20130204214538146 lockFactory=org.apache.lucene.store.NativeFSLockFactory@408f1117),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 20665 T894 C92 P58727 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 20666 T894 C92 P58727 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty3
[junit4:junit4]   2> 20666 T894 C92 P58727 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 20667 T894 C92 P58727 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty3/index.20130204214538146
[junit4:junit4]   2> 20667 T894 C92 P58727 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty3/index
[junit4:junit4]   2> 20667 T894 C92 P58727 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 20668 T894 C92 P58727 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 20668 T894 C92 P58727 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 20668 T894 C92 P58727 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 20670 T894 C92 P58727 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 21112 T843 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 21112 T820 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 21113 T843 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:58735_i",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:58735/i"}
[junit4:junit4]   2> 21118 T843 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:58727_i",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:58727/i"}
[junit4:junit4]   2> 21136 T876 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 21136 T907 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 21136 T891 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 21136 T842 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 21136 T848 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 21136 T861 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 22115 T820 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 23118 T820 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C93_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:58735_i, base_url=http://127.0.0.1:58735/i}
[junit4:junit4]   2> 24111 T910 C93 P58735 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:58724/i/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 24111 T910 C93 P58735 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:58735/i START replicas=[http://127.0.0.1:58724/i/collection1/] nUpdates=100
[junit4:junit4]   2> 24112 T910 C93 P58735 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 24112 T910 C93 P58735 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 24112 T910 C93 P58735 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 24112 T910 C93 P58735 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 24113 T910 C93 P58735 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 24113 T910 C93 P58735 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:58724/i/collection1/. core=collection1
[junit4:junit4]   2> 24114 T910 C93 P58735 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 24121 T820 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> ASYNC  NEW_CORE C95 name=collection1 org.apache.solr.core.SolrCore@c94f09b url=http://127.0.0.1:58724/i/collection1 node=127.0.0.1:58724_i C95_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:58724_i, base_url=http://127.0.0.1:58724/i, leader=true}
[junit4:junit4]   2> 24125 T867 C95 P58724 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 24132 T868 C95 P58724 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 24133 T868 C95 P58724 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty2
[junit4:junit4]   2> 24137 T868 C95 P58724 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4d130642),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 24138 T868 C95 P58724 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 24143 T868 C95 P58724 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4d130642),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4d130642),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 24144 T868 C95 P58724 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 24145 T868 C95 P58724 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty2
[junit4:junit4]   2> 24145 T868 C95 P58724 oass.SolrIndexSearcher.<init> Opening Searcher@3852e669 realtime
[junit4:junit4]   2> 24146 T868 C95 P58724 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 24146 T868 C95 P58724 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 14
[junit4:junit4]   2> 24147 T910 C93 P58735 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 24148 T910 C93 P58735 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 24150 T869 C95 P58724 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 24151 T869 C95 P58724 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 24152 T910 C93 P58735 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 24152 T910 C93 P58735 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 24152 T910 C93 P58735 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 24154 T869 C95 P58724 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty2
[junit4:junit4]   2> 24155 T869 C95 P58724 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty2/index
[junit4:junit4]   2> 24156 T869 C95 P58724 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=2 
[junit4:junit4]   2> 24156 T910 C93 P58735 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 24158 T910 C93 P58735 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty4/index.20130204214541692 forceNew:false
[junit4:junit4]   2> 24158 T910 C93 P58735 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty4
[junit4:junit4]   2> 24159 T910 C93 P58735 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty4/index.20130204214541692 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7e89b998) fullCopy=true
[junit4:junit4]   2> 24163 T869 C95 P58724 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> 24165 T910 C93 P58735 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty4/index
[junit4:junit4]   2> 24166 T910 C93 P58735 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 24171 T910 C93 P58735 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130204214541692
[junit4:junit4]   2> 24172 T910 C93 P58735 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty4
[junit4:junit4]   2> 24173 T910 C93 P58735 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty4
[junit4:junit4]   2> 24173 T910 C93 P58735 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty4/index/ new=./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty4/index.20130204214541692
[junit4:junit4]   2> 24176 T910 C93 P58735 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty4/index.20130204214541692 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7e89b998),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 24177 T910 C93 P58735 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 24178 T910 C93 P58735 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty4
[junit4:junit4]   2> 24178 T910 C93 P58735 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 24179 T910 C93 P58735 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 24179 T910 C93 P58735 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 24180 T910 C93 P58735 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty4/index.20130204214541692
[junit4:junit4]   2> 24180 T910 C93 P58735 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty4
[junit4:junit4]   2> 24184 T910 C93 P58735 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty4/index.20130204214541692 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7e89b998),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 24184 T910 C93 P58735 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 24184 T910 C93 P58735 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 24185 T910 C93 P58735 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty4
[junit4:junit4]   2> 24185 T910 C93 P58735 oass.SolrIndexSearcher.<init> Opening Searcher@5f1e37eb main
[junit4:junit4]   2> 24187 T909 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5f1e37eb main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 24188 T909 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty4/index
[junit4:junit4]   2> 24188 T910 C93 P58735 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 24209 T910 C93 P58735 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty4/index.20130204214541692 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7e89b998),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty4/index.20130204214541692 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7e89b998),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 24210 T910 C93 P58735 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 24242 T910 C93 P58735 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty4
[junit4:junit4]   2> 24243 T910 C93 P58735 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 24243 T910 C93 P58735 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty4/index.20130204214541692
[junit4:junit4]   2> 24243 T910 C93 P58735 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty4/index
[junit4:junit4]   2> 24243 T910 C93 P58735 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 24244 T910 C93 P58735 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 24244 T910 C93 P58735 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 24244 T910 C93 P58735 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 24250 T910 C93 P58735 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 25123 T820 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 25649 T843 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 25650 T843 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:58735_i",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:58735/i"}
[junit4:junit4]   2> 25677 T861 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 25677 T907 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 25677 T842 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 25677 T891 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 25677 T848 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 25677 T876 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 26127 T820 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 26128 T820 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4]   2> ASYNC  NEW_CORE C96 name=collection1 org.apache.solr.core.SolrCore@2410a383 url=http://127.0.0.1:58702/i/collection1 node=127.0.0.1:58702_i C96_STATE=coll:control_collection core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:58702_i, base_url=http://127.0.0.1:58702/i, leader=true}
[junit4:junit4]   2> 26161 T832 C96 P58702 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/control/data
[junit4:junit4]   2> 26166 T832 C96 P58702 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@790d9a3a),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 26166 T832 C96 P58702 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 26168 T832 C96 P58702 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/control/data
[junit4:junit4]   2> 26169 T832 C96 P58702 oass.SolrIndexSearcher.<init> Opening Searcher@5955cd23 realtime
[junit4:junit4]   2> 26169 T832 C96 P58702 /update {wt=javabin&version=2} {deleteByQuery=*:* (-1426101049695928320)} 0 14
[junit4:junit4]   2> 26186 T870 C95 P58724 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty2
[junit4:junit4]   2> 26242 T855 C94 P58721 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty1
[junit4:junit4]   2>  C93_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:58735_i, base_url=http://127.0.0.1:58735/i}
[junit4:junit4]   2> 26242 T897 C93 P58735 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty4
[junit4:junit4]   2> 26243 T897 C93 P58735 /update {update.distrib=FROMLEADER&_version_=-1426101049721094144&update.from=http://127.0.0.1:58724/i/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1426101049721094144)} 0 19
[junit4:junit4]   2>  C92_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:58727_i, base_url=http://127.0.0.1:58727/i}
[junit4:junit4]   2> 26256 T881 C92 P58727 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360035917540/jetty3
[junit4:junit4]   2> 26258 T881 C92 P58727 /update {update.distrib=FROMLEADER&_version_=-1426101049749405696&update.from=http://127.0.0.1:58721/i/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1426101049749405696)} 0 4
[junit4:junit4]   2> 26259 T855 C94 P58721 /update {update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1426101049749405696)} 0 48
[junit4:junit4]   2> 26259 T870 C95 P58724 /update {wt=javabin&version=2} {deleteByQuery=*:* (-1426101049721094144)} 0 86
[junit4:junit4]   2> 26269 T832 C96 P58702 /update {wt=javabin&version=2} {add=[1 (1426101049804980224)]} 0 6
[junit4:junit4]   2> 26296 T881 C92 P58727 /update {distrib.from=http://127.0.0.1:58721/i/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1426101049813368832)]} 0 6
[junit4:junit4]   2> 26297 T856 C94 P58721 /update {wt=javabin&version=2} {add=[1 (1426101049813368832)]} 0 25
[junit4:junit4]   2> 26304 T832 C96 P58702 /update {wt=javabin&version=2} {add=[2 (1426101049844826112)]} 0 3
[junit4:junit4]   2> 26320 T897 C93 P58735 /update {distrib.from=http://127.0.0.1:58724/i/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1426101049851117568)]} 0 5
[junit4:junit4]   2> 26321 T870 C95 P58724 /update {wt=javabin&version=2} {add=[2 (1426101049851117568)]} 0 14
[junit4:junit4]   2> 26327 T832 C96 P58702 /update {wt=javabin&version=2} {add=[3 (1426101049868943360)]} 0 3
[junit4:junit4]   2> 26347 T897 C93 P58735 /update {distrib.from=http://127.0.0.1:58724/i/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[3 (1426101049883623424)]} 0 3
[junit4:junit4]   2> 26347 T871 C95 P58724 /update {distrib.from=http://127.0.0.1:58721/i/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[3 (1426101049883623424)]} 0 9
[junit4:junit4]   2> 26348 T856 C94 P58721 /update {wt=javabin&version=2} {add=[3]} 0 18
[junit4:junit4]   2> 26354 T832 C96 P58702 /update {wt=javabin&version=2} {add=[4 (1426101049897254912)]} 0 3
[junit4:junit4]   2> 26365 T881 C92 P58727 /update {distrib.from=http://127.0.0.1:58721/i/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[4 (1426101049903546368)]} 0 3
[junit4:junit4]   2> 26366 T856 C94 P58721 /update {wt=javabin&version=2} {add=[4 (1426101049903546368)]} 0 9
[junit4:junit4]   2> 26372 T832 C96 P58702 /update {wt=javabin&version=2} {add=[5 (1426101049916129280)]} 0 3
[junit4:junit4]   2> 26383 T897 C93 P58735 /update {distrib.from=http://127.0.0.1:58724/i/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[5 (1426101049922420736)]} 0 3
[junit4:junit4]   2> 26383 T870 C95 P58724 /update {wt=javabin&version=2} {add=[5 (1426101049922420736)]} 0 8
[junit4:junit4]   2> 26390 T832 C96 P58702 /update {wt=javabin&version=2} {add=[6 (1426101049935003648)]} 0 3
[junit4:junit4]   2> 26401 T897 C93 P58735 /update {distrib.from=http://127.0.0.1:58724/i/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[6 (1426101049941295104)]} 0 3
[junit4:junit4]   2> 26402 T870 C95 P58724 /update {wt=javabin&version=2} {add=[6 (1426101049941295104)]} 0 9
[junit4:junit4]   2> 26408 T832 C96 P58702 /update {wt=javabin&version=2} {add=[7 (1426101049953878016)]} 0 3
[junit4:junit4]   2> 26422 T897 C93 P58735 /update {distrib.from=http://127.0.0.1:58724/i/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[7 (1426101049963315200)]} 0 3
[junit4:junit4]   2> 26423 T871 C95 P58724 /update {distrib.from=http://127.0.0.1:58721/i/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[7 (1426101049963315200)]} 0 9
[junit4:junit4]   2> 26423 T856 C94 P58721 /update {wt=javabin&version=2} {add=[7]} 0 12
[junit4:junit4]   2> 26430 T832 C96 P58702 /update {wt=javabin&version=2} {add=[8 (1426101049976946688)]} 0 3
[junit4:junit4]   2> 26445 T881 C92 P58727 /update {distrib.from=http://127.0.0.1:58721/i/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[8 (1426101049986383872)]} 0 3
[junit4:junit4]   2> 26446 T855 C94 P58721 /update {distrib.from=http://127.0.0.1:58724/i/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[8 (1426101049986383872)]} 0 10
[junit4:junit4]   2> 26446 T870 C95 P58724 /update {wt=javabin&version=2} {add=[8]} 0 13
[junit4:junit4]   2> 26453 T832 C96 P58702 /update {wt=javabin&version=2} {add=[9 (1426101050001063936)]} 0 3
[junit4:junit4]   2> 26467 T897 C93 P58735 /update {distrib.from=http://127.0.0.1:58724/i/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[9 (1426101050010501120)]} 0 3
[junit4:junit4]   2> 26467 T871 C95 P58724 /update {distrib.from=http://127.0.0.1:58721/i/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[9 (1426101050010501120)]} 0 8
[junit4:junit4]   2> 26468 T856 C94 P58721 /update {wt=javabin&version=2} {add=[9]} 0 12
[junit4:junit4]   2> 26505 T832 C96 P58702 /update {wt=javabin&version=2} {add=[10 (1426101050023084032)]} 0 34
[junit4:junit4]   2> 26519 T881 C92 P58727 /update {distrib.from=http://127.0.0.1:58721/i/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10 (1426101050065027072)]} 0 3
[junit4:junit4]   2> 26520 T855 C94 P58721 /update {distrib.from=http://127.0.0.1:58724/i/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[10 (1426101050065027072)]} 0 9
[junit4:junit4]   2> 26520 T870 C95 P58724 /update {wt=javabin&version=2} {add=[10]} 0 12
[junit4:junit4]   2> 26529 T832 C96 P58702 /update {wt=javabin&version=2} {add=[11 (1426101050078658560)]} 0 5
[junit4:junit4]   2> 26540 T881 C92 P58727 /update {distrib.from=http://127.0.0.1:58721/i/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[11 (1426101050087047168)]} 0 3
[junit4:junit4]   2> 26541 T856 C94 P58721 /update {wt=javabin&version=2} {add=[11 (1426101050087047168)]} 0 9
[junit4:junit4]   2> 26547 T832 C96 P58702 /update {wt=javabin&version=2} {add=[12 (1426101050099630080)]} 0 3
[junit4:junit4]   2> 26558 T881 C92 P58727 /update {distrib.from=http://127.0.0.1:58721/i/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[12 (1426101050105921536)]} 0 3
[junit4:junit4]   2> 26558 T856 C94 P58721 /update {wt=javabin&version=2} {add=[12 (1426101050105921536)]} 0 8
[junit4:junit4]   2> 26565 T832 C96 P58702 /update {wt=javabin&version=2} {add=[13 (1426101050118504448)]} 0 3
[junit4:junit4]   2> 26576 T881 C92 P58727 /update {distrib.fr

[...truncated too long message...]

	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  24) Thread[id=901, name=qtp1078494859-901, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at java.net.SocketInputStream.socketRead0(Native Method)
[junit4:junit4]   2> 	        at java.net.SocketInputStream.read(SocketInputStream.java:150)
[junit4:junit4]   2> 	        at java.net.SocketInputStream.read(SocketInputStream.java:121)
[junit4:junit4]   2> 	        at org.eclipse.jetty.io.ByteArrayBuffer.readFrom(ByteArrayBuffer.java:375)
[junit4:junit4]   2> 	        at org.eclipse.jetty.io.bio.StreamEndPoint.fill(StreamEndPoint.java:141)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.fill(SocketConnector.java:227)
[junit4:junit4]   2> 	        at org.eclipse.jetty.http.HttpParser.fill(HttpParser.java:1035)
[junit4:junit4]   2> 	        at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:280)
[junit4:junit4]   2> 	        at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  25) Thread[id=906, name=TEST-BasicDistributedZkTest.testDistribSearch-seed#[38B502A79FD73A82]-SendThread(localhost:58698), state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:82)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
[junit4:junit4]   2> 	  26) Thread[id=896, name=qtp1078494859-896 Acceptor0 SocketConnector@127.0.0.1:58735, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at java.net.PlainSocketImpl.socketAccept(Native Method)
[junit4:junit4]   2> 	        at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
[junit4:junit4]   2> 	        at java.net.ServerSocket.implAccept(ServerSocket.java:522)
[junit4:junit4]   2> 	        at java.net.ServerSocket.accept(ServerSocket.java:490)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.bio.SocketConnector.accept(SocketConnector.java:117)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  27) Thread[id=878, name=searcherExecutor-616-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  28) Thread[id=850, name=qtp454322762-850 Acceptor0 SocketConnector@127.0.0.1:58721, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at java.net.PlainSocketImpl.socketAccept(Native Method)
[junit4:junit4]   2> 	        at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
[junit4:junit4]   2> 	        at java.net.ServerSocket.implAccept(ServerSocket.java:522)
[junit4:junit4]   2> 	        at java.net.ServerSocket.accept(ServerSocket.java:490)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.bio.SocketConnector.accept(SocketConnector.java:117)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> NOTE: test params are: codec=Lucene42: {n_f1=PostingsFormat(name=NestedPulsing), foo_b=PostingsFormat(name=MockSep), foo_d=MockFixedIntBlock(blockSize=585), foo_f=PostingsFormat(name=MockSep), n_tl1=PostingsFormat(name=NestedPulsing), n_d1=Pulsing41(freqCutoff=14 minBlockSize=42 maxBlockSize=92), rnd_b=MockFixedIntBlock(blockSize=585), intDefault=PostingsFormat(name=NestedPulsing), n_td1=PostingsFormat(name=NestedPulsing), timestamp=Pulsing41(freqCutoff=14 minBlockSize=42 maxBlockSize=92), id=MockFixedIntBlock(blockSize=585), range_facet_sl=MockFixedIntBlock(blockSize=585), range_facet_si=Pulsing41(freqCutoff=14 minBlockSize=42 maxBlockSize=92), oddField_s=MockFixedIntBlock(blockSize=585), multiDefault=Pulsing41(freqCutoff=14 minBlockSize=42 maxBlockSize=92), n_tf1=Pulsing41(freqCutoff=14 minBlockSize=42 maxBlockSize=92), n_dt1=PostingsFormat(name=NestedPulsing), n_ti1=MockFixedIntBlock(blockSize=585), range_facet_l=Pulsing41(freqCutoff=14 minBlockSize=42 maxBlockSize=92), text=PostingsFormat(name=MockSep), _version_=PostingsFormat(name=NestedPulsing), SubjectTerms_mfacet=PostingsFormat(name=NestedPulsing), a_t=Pulsing41(freqCutoff=14 minBlockSize=42 maxBlockSize=92), n_tdt1=PostingsFormat(name=NestedPulsing), other_tl1=PostingsFormat(name=NestedPulsing), n_l1=Pulsing41(freqCutoff=14 minBlockSize=42 maxBlockSize=92), a_si=PostingsFormat(name=NestedPulsing)}, sim=DefaultSimilarity, locale=hr_HR, timezone=Etc/GMT+6
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=37,free=171959472,total=308281344
[junit4:junit4]   2> NOTE: All tests run in this JVM: [DirectUpdateHandlerOptimizeTest, TestDocumentBuilder, PluginInfoTest, IndexBasedSpellCheckerTest, TestQuerySenderNoQuery, ResourceLoaderTest, NumericFieldsTest, PreAnalyzedFieldTest, DateMathParserTest, TestPerFieldSimilarity, TestGroupingSearch, TestLazyCores, TestSolrIndexConfig, AutoCommitTest, TestCharFilters, TestSolr4Spatial, TestIndexSearcher, MinimalSchemaTest, SampleTest, SpellPossibilityIteratorTest, TestNumberUtils, WordBreakSolrSpellCheckerTest, AnalysisAfterCoreReloadTest, CoreContainerCoreInitFailuresTest, FieldAnalysisRequestHandlerTest, DistributedSpellCheckComponentTest, SolrCoreCheckLockOnStartupTest, DateFieldTest, SolrRequestParserTest, ZkCLITest, TestFuzzyAnalyzedSuggestions, JSONWriterTest, ScriptEngineTest, ClusterStateUpdateTest, FileBasedSpellCheckerTest, TestConfig, TestStressVersions, TestReload, CoreAdminHandlerTest, TestPHPSerializedResponseWriter, JsonLoaderTest, HighlighterConfigTest, SpatialFilterTest, DistributedTermsComponentTest, MBeansHandlerTest, ZkControllerTest, SpellCheckCollatorTest, DefaultValueUpdateProcessorTest, TestPseudoReturnFields, NoCacheHeaderTest, RequestHandlersTest, TestExtendedDismaxParser, NotRequiredUniqueKeyTest, TestSearchPerf, SearchHandlerTest, QueryElevationComponentTest, CurrencyFieldTest, TestQueryTypes, DistanceFunctionTest, BasicDistributedZkTest]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=BasicDistributedZkTest -Dtests.seed=38B502A79FD73A82 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=hr_HR -Dtests.timezone=Etc/GMT+6 -Dtests.file.encoding=UTF-8
[junit4:junit4] ERROR   0.00s J0 | BasicDistributedZkTest (suite) <<<
[junit4:junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([38B502A79FD73A82]:0)
[junit4:junit4] Completed on J0 in 7223.58s, 1 test, 2 errors <<< FAILURES!

[...truncated 11 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:346: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/common-build.xml:428: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:1176: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:841: There were test failures: 249 suites, 1038 tests, 1 suite-level error, 1 error, 23 ignored (5 assumptions)

Total time: 155 minutes 8 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



Re: [JENKINS] Lucene-Solr-Tests-trunk-java7 - Build # 3708 - Still Failing

Posted by Mark Miller <ma...@gmail.com>.
Tripped a one minute so timeout - guess I can up it to 2 minutes.

- Mark

On Feb 4, 2013, at 6:26 PM, Apache Jenkins Server <je...@builds.apache.org> wrote:

> Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-java7/3708/
> 
> 1 tests failed.
> FAILED:  org.apache.solr.cloud.BasicDistributedZkTest.testDistribSearch
> 
> Error Message:
> Captured an uncaught exception in thread: Thread[id=935, name=testExecutor-345-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
> 
> Stack Trace:
> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=935, name=testExecutor-345-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
> Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:30255/sp_ffw/f
> 	at __randomizedtesting.SeedInfo.seed([715CCD8973136CF0]:0)
> 	at org.apache.solr.cloud.BasicDistributedZkTest$1.run(BasicDistributedZkTest.java:658)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> 	at java.lang.Thread.run(Thread.java:722)
> Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:30255/sp_ffw/f
> 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:412)
> 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181)
> 	at org.apache.solr.cloud.BasicDistributedZkTest$1.run(BasicDistributedZkTest.java:656)
> 	... 3 more
> Caused by: java.net.SocketTimeoutException: Read timed out
> 	at java.net.SocketInputStream.socketRead0(Native Method)
> 	at java.net.SocketInputStream.read(SocketInputStream.java:150)
> 	at java.net.SocketInputStream.read(SocketInputStream.java:121)
> 	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:149)
> 	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:111)
> 	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:264)
> 	at org.apache.http.impl.conn.DefaultResponseParser.parseHead(DefaultResponseParser.java:98)
> 	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:252)
> 	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:282)
> 	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:247)
> 	at org.apache.http.impl.conn.AbstractClientConnAdapter.receiveResponseHeader(AbstractClientConnAdapter.java:216)
> 	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:298)
> 	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
> 	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:647)
> 	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:464)
> 	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
> 	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
> 	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:732)
> 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:353)
> 	... 5 more
> 
> 
> 
> 
> Build Log:
> [...truncated 8817 lines...]
> [junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
> [junit4:junit4]   2> 0 T428 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /sp_ffw/f
> [junit4:junit4]   2> 4 T428 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
> [junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZkTest-1360019516837
> [junit4:junit4]   2> 11 T428 oasc.ZkTestServer.run STARTING ZK TEST SERVER
> [junit4:junit4]   2> 15 T429 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
> [junit4:junit4]   2> 73 T429 oaz.Environment.logEnv Server environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT
> [junit4:junit4]   2> 74 T429 oaz.Environment.logEnv Server environment:host.name=lucene.zones.apache.org
> [junit4:junit4]   2> 74 T429 oaz.Environment.logEnv Server environment:java.version=1.7.0_09
> [junit4:junit4]   2> 74 T429 oaz.Environment.logEnv Server environment:java.vendor=Oracle Corporation
> [junit4:junit4]   2> 74 T429 oaz.Environment.logEnv Server environment:java.home=/usr/local/openjdk7/jre
> [junit4:junit4]   2> 74 T429 oaz.Environment.logEnv Server environment:java.class.path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/classes/test:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/codecs/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-solrj/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/analysis/common/lucene-analyzers-common-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/highlighter/lucene-highlighter-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/memory/lucene-memory-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/misc/lucene-misc-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/spatial/lucene-spatial-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/suggest/lucene-suggest-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/grouping/lucene-grouping-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/queries/lucene-queries-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/queryparser/lucene-queryparser-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/cglib-nodep-2.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/commons-cli-1.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/commons-codec-1.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/commons-fileupload-1.2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/commons-lang-2.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/easymock-3.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/guava-13.0.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/javax.servlet-api-3.0.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/objenesis-1.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/spatial4j-0.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/commons-io-2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/httpclient-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/httpcore-4.1.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/httpmime-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/jcl-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/slf4j-api-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/slf4j-jdk14-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/wstx-asl-3.2.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/zookeeper-3.4.5.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-continuation-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-deploy-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-http-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-io-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-jmx-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-security-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-server-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-servlet-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-util-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-webapp-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-xml-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/servlet-api-3.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/example-DIH/solr/db/lib/derby-10.9.1.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/test-framework/lib/junit-4.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/test-framework/lib/randomizedtesting-runner-2.0.8.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/home/hudson/.ant/lib/ivy-2.2.0.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/usr/local/openjdk7/lib/tools.jar:/home/hudson/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.0.8.jar
> [junit4:junit4]   2> 75 T429 oaz.Environment.logEnv Server environment:java.library.path=/usr/local/openjdk7/jre/lib/amd64/server:/usr/local/openjdk7/jre/lib/amd64:/usr/local/openjdk7/jre/../lib/amd64:/usr/local/openjdk6/jre/lib/amd64/server:/usr/local/openjdk6/jre/lib/amd64:/usr/local/openjdk6/jre/../lib/amd64:/usr/java/packages/lib/amd64:/lib:/usr/lib:/usr/local/lib
> [junit4:junit4]   2> 75 T429 oaz.Environment.logEnv Server environment:java.io.tmpdir=.
> [junit4:junit4]   2> 75 T429 oaz.Environment.logEnv Server environment:java.compiler=<NA>
> [junit4:junit4]   2> 75 T429 oaz.Environment.logEnv Server environment:os.name=FreeBSD
> [junit4:junit4]   2> 75 T429 oaz.Environment.logEnv Server environment:os.arch=amd64
> [junit4:junit4]   2> 76 T429 oaz.Environment.logEnv Server environment:os.version=9.0-RELEASE
> [junit4:junit4]   2> 76 T429 oaz.Environment.logEnv Server environment:user.name=hudson
> [junit4:junit4]   2> 76 T429 oaz.Environment.logEnv Server environment:user.home=/home/hudson
> [junit4:junit4]   2> 76 T429 oaz.Environment.logEnv Server environment:user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
> [junit4:junit4]   2> 109 T429 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
> [junit4:junit4]   2> 109 T429 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
> [junit4:junit4]   2> 109 T429 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
> [junit4:junit4]   2> 115 T429 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
> [junit4:junit4]   2> 116 T428 oasc.ZkTestServer.run start zk server on port:45780
> [junit4:junit4]   2> 179 T428 oaz.Environment.logEnv Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT
> [junit4:junit4]   2> 180 T428 oaz.Environment.logEnv Client environment:host.name=lucene.zones.apache.org
> [junit4:junit4]   2> 180 T428 oaz.Environment.logEnv Client environment:java.version=1.7.0_09
> [junit4:junit4]   2> 180 T428 oaz.Environment.logEnv Client environment:java.vendor=Oracle Corporation
> [junit4:junit4]   2> 180 T428 oaz.Environment.logEnv Client environment:java.home=/usr/local/openjdk7/jre
> [junit4:junit4]   2> 181 T428 oaz.Environment.logEnv Client environment:java.class.path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/classes/test:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/codecs/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-solrj/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/analysis/common/lucene-analyzers-common-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/highlighter/lucene-highlighter-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/memory/lucene-memory-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/misc/lucene-misc-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/spatial/lucene-spatial-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/suggest/lucene-suggest-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/grouping/lucene-grouping-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/queries/lucene-queries-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/queryparser/lucene-queryparser-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/cglib-nodep-2.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/commons-cli-1.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/commons-codec-1.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/commons-fileupload-1.2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/commons-lang-2.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/easymock-3.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/guava-13.0.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/javax.servlet-api-3.0.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/objenesis-1.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/spatial4j-0.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/commons-io-2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/httpclient-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/httpcore-4.1.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/httpmime-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/jcl-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/slf4j-api-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/slf4j-jdk14-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/wstx-asl-3.2.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/zookeeper-3.4.5.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-continuation-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-deploy-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-http-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-io-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-jmx-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-security-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-server-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-servlet-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-util-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-webapp-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-xml-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/servlet-api-3.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/example-DIH/solr/db/lib/derby-10.9.1.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/test-framework/lib/junit-4.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/test-framework/lib/randomizedtesting-runner-2.0.8.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/home/hudson/.ant/lib/ivy-2.2.0.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/usr/local/openjdk7/lib/tools.jar:/home/hudson/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.0.8.jar
> [junit4:junit4]   2> 182 T428 oaz.Environment.logEnv Client environment:java.library.path=/usr/local/openjdk7/jre/lib/amd64/server:/usr/local/openjdk7/jre/lib/amd64:/usr/local/openjdk7/jre/../lib/amd64:/usr/local/openjdk6/jre/lib/amd64/server:/usr/local/openjdk6/jre/lib/amd64:/usr/local/openjdk6/jre/../lib/amd64:/usr/java/packages/lib/amd64:/lib:/usr/lib:/usr/local/lib
> [junit4:junit4]   2> 182 T428 oaz.Environment.logEnv Client environment:java.io.tmpdir=.
> [junit4:junit4]   2> 182 T428 oaz.Environment.logEnv Client environment:java.compiler=<NA>
> [junit4:junit4]   2> 183 T429 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/zookeeper/server1/data/version-2/snapshot.0
> [junit4:junit4]   2> 183 T428 oaz.Environment.logEnv Client environment:os.name=FreeBSD
> [junit4:junit4]   2> 183 T428 oaz.Environment.logEnv Client environment:os.arch=amd64
> [junit4:junit4]   2> 183 T428 oaz.Environment.logEnv Client environment:os.version=9.0-RELEASE
> [junit4:junit4]   2> 183 T428 oaz.Environment.logEnv Client environment:user.name=hudson
> [junit4:junit4]   2> 184 T428 oaz.Environment.logEnv Client environment:user.home=/home/hudson
> [junit4:junit4]   2> 184 T428 oaz.Environment.logEnv Client environment:user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
> [junit4:junit4]   2> 187 T428 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45780 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@b2da32d
> [junit4:junit4]   2> 246 T428 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4]   2> 263 T432 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45780. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4]   2> 266 T432 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45780, initiating session
> [junit4:junit4]   2> 267 T430 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:21112
> [junit4:junit4]   2> 284 T430 oazs.NIOServerCnxn.doIO WARNING Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running
> [junit4:junit4]   2> 285 T430 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:21112 (no session established for client)
> [junit4:junit4]   2> 285 T432 oaz.ClientCnxn$SendThread.run Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
> [junit4:junit4]   2> 2163 T432 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45780. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4]   2> 2164 T432 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45780, initiating session
> [junit4:junit4]   2> 2164 T430 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:19269
> [junit4:junit4]   2> 2178 T430 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:19269
> [junit4:junit4]   2> 2194 T434 oazsp.FileTxnLog.append Creating new log file: log.1
> [junit4:junit4]   2> 2235 T434 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca77aee720000 with negotiated timeout 10000 for client /140.211.11.196:19269
> [junit4:junit4]   2> 2236 T432 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45780, sessionid = 0x13ca77aee720000, negotiated timeout = 10000
> [junit4:junit4]   2> 2240 T433 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b2da32d name:ZooKeeperConnection Watcher:127.0.0.1:45780 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4]   2> 2240 T428 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4]   2> 2245 T428 oascc.SolrZkClient.makePath makePath: /solr
> [junit4:junit4]   2> 2329 T435 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca77aee720000
> [junit4:junit4]   2> 2331 T433 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4]   2> 2331 T428 oaz.ZooKeeper.close Session: 0x13ca77aee720000 closed
> [junit4:junit4]   2> 2331 T430 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:19269 which had sessionid 0x13ca77aee720000
> [junit4:junit4]   2> 2331 T428 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45780/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@79ba1ec8
> [junit4:junit4]   2> 2332 T428 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4]   2> 2333 T436 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45780. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4]   2> 2333 T436 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45780, initiating session
> [junit4:junit4]   2> 2334 T430 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:49718
> [junit4:junit4]   2> 2334 T430 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:49718
> [junit4:junit4]   2> 2336 T434 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca77aee720001 with negotiated timeout 10000 for client /140.211.11.196:49718
> [junit4:junit4]   2> 2336 T436 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45780, sessionid = 0x13ca77aee720001, negotiated timeout = 10000
> [junit4:junit4]   2> 2337 T437 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@79ba1ec8 name:ZooKeeperConnection Watcher:127.0.0.1:45780/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4]   2> 2337 T428 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4]   2> 2343 T428 oascc.SolrZkClient.makePath makePath: /collections/collection1
> [junit4:junit4]   2> 2349 T428 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
> [junit4:junit4]   2> 2360 T428 oascc.SolrZkClient.makePath makePath: /collections/control_collection
> [junit4:junit4]   2> 2364 T428 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
> [junit4:junit4]   2> 2369 T428 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
> [junit4:junit4]   2> 2397 T428 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
> [junit4:junit4]   2> 2413 T428 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
> [junit4:junit4]   2> 2414 T428 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
> [junit4:junit4]   2> 2521 T428 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
> [junit4:junit4]   2> 2522 T428 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
> [junit4:junit4]   2> 2528 T428 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
> [junit4:junit4]   2> 2529 T428 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
> [junit4:junit4]   2> 2535 T428 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
> [junit4:junit4]   2> 2535 T428 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
> [junit4:junit4]   2> 2542 T428 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
> [junit4:junit4]   2> 2542 T428 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
> [junit4:junit4]   2> 2549 T428 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
> [junit4:junit4]   2> 2549 T428 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
> [junit4:junit4]   2> 2555 T428 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
> [junit4:junit4]   2> 2556 T428 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
> [junit4:junit4]   2> 2562 T428 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
> [junit4:junit4]   2> 2563 T428 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
> [junit4:junit4]   2> 2569 T435 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca77aee720001
> [junit4:junit4]   2> 2570 T437 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4]   2> 2570 T428 oaz.ZooKeeper.close Session: 0x13ca77aee720001 closed
> [junit4:junit4]   2> 2570 T430 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:49718 which had sessionid 0x13ca77aee720001
> [junit4:junit4]   2> 2769 T428 oejs.Server.doStart jetty-8.1.8.v20121106
> [junit4:junit4]   2> 2773 T428 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:64828
> [junit4:junit4]   2> 2774 T428 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4:junit4]   2> 2774 T428 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4:junit4]   2> 2775 T428 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360019519403
> [junit4:junit4]   2> 2775 T428 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360019519403/solr.xml
> [junit4:junit4]   2> 2776 T428 oasc.CoreContainer.<init> New CoreContainer 1882182574
> [junit4:junit4]   2> 2777 T428 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360019519403/'
> [junit4:junit4]   2> 2777 T428 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360019519403/'
> [junit4:junit4]   2> 2839 T428 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
> [junit4:junit4]   2> 2839 T428 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4:junit4]   2> 2840 T428 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4:junit4]   2> 2840 T428 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4:junit4]   2> 2841 T428 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4:junit4]   2> 2841 T428 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4:junit4]   2> 2842 T428 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4:junit4]   2> 2842 T428 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4:junit4]   2> 2843 T428 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4:junit4]   2> 2843 T428 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
> [junit4:junit4]   2> 2871 T428 oasc.CoreContainer.load Registering Log Listener
> [junit4:junit4]   2> 2895 T428 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:45780/solr
> [junit4:junit4]   2> 2921 T428 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4:junit4]   2> 2922 T428 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45780 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@31415815
> [junit4:junit4]   2> 2923 T428 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4]   2> 2923 T447 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45780. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4]   2> 2924 T447 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45780, initiating session
> [junit4:junit4]   2> 2924 T430 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:24071
> [junit4:junit4]   2> 2925 T430 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:24071
> [junit4:junit4]   2> 2926 T434 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca77aee720002 with negotiated timeout 20000 for client /140.211.11.196:24071
> [junit4:junit4]   2> 2926 T447 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45780, sessionid = 0x13ca77aee720002, negotiated timeout = 20000
> [junit4:junit4]   2> 2927 T448 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@31415815 name:ZooKeeperConnection Watcher:127.0.0.1:45780 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4]   2> 2927 T428 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4]   2> 2929 T435 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca77aee720002
> [junit4:junit4]   2> 2930 T448 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4]   2> 2930 T430 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:24071 which had sessionid 0x13ca77aee720002
> [junit4:junit4]   2> 2930 T428 oaz.ZooKeeper.close Session: 0x13ca77aee720002 closed
> [junit4:junit4]   2> 2936 T428 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
> [junit4:junit4]   2> 2955 T428 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45780/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@56fe2145
> [junit4:junit4]   2> 2956 T428 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4]   2> 2957 T449 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45780. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4]   2> 2957 T449 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45780, initiating session
> [junit4:junit4]   2> 2958 T430 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:49673
> [junit4:junit4]   2> 2958 T430 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:49673
> [junit4:junit4]   2> 2960 T434 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca77aee720003 with negotiated timeout 20000 for client /140.211.11.196:49673
> [junit4:junit4]   2> 2960 T449 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45780, sessionid = 0x13ca77aee720003, negotiated timeout = 20000
> [junit4:junit4]   2> 2960 T450 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@56fe2145 name:ZooKeeperConnection Watcher:127.0.0.1:45780/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4]   2> 2960 T428 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4]   2> 2982 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 2994 T428 oascc.SolrZkClient.makePath makePath: /live_nodes
> [junit4:junit4]   2> 2998 T428 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:64828_sp_ffw%2Ff
> [junit4:junit4]   2> 3004 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720003 type:delete cxid:0x7 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:64828_sp_ffw%2Ff Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:64828_sp_ffw%2Ff
> [junit4:junit4]   2> 3006 T428 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:64828_sp_ffw%2Ff
> [junit4:junit4]   2> 3012 T428 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
> [junit4:junit4]   2> 3047 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720003 type:delete cxid:0x15 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
> [junit4:junit4]   2> 3048 T428 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
> [junit4:junit4]   2> 3053 T428 oasc.Overseer.start Overseer (id=89130239068864515-127.0.0.1:64828_sp_ffw%2Ff-n_0000000000) starting
> [junit4:junit4]   2> 3055 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720003 type:create cxid:0x1a zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 3086 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720003 type:create cxid:0x1b zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 3088 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720003 type:create cxid:0x1c zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 3102 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720003 type:create cxid:0x1d zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 3104 T452 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
> [junit4:junit4]   2> 3115 T428 oascc.SolrZkClient.makePath makePath: /clusterstate.json
> [junit4:junit4]   2> 3119 T428 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
> [junit4:junit4]   2> 3144 T451 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
> [junit4:junit4]   2> 3161 T453 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360019519403/collection1
> [junit4:junit4]   2> 3161 T453 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
> [junit4:junit4]   2> 3162 T453 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4:junit4]   2> 3162 T453 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
> [junit4:junit4]   2> 3165 T453 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360019519403/collection1/'
> [junit4:junit4]   2> 3166 T453 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360019519403/collection1/lib/README' to classloader
> [junit4:junit4]   2> 3167 T453 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360019519403/collection1/lib/classes/' to classloader
> [junit4:junit4]   2> 3246 T453 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4:junit4]   2> 3326 T453 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4:junit4]   2> 3427 T453 oass.IndexSchema.readSchema Reading Solr Schema
> [junit4:junit4]   2> 3435 T453 oass.IndexSchema.readSchema Schema name=test
> [junit4:junit4]   2> 4143 T453 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4:junit4]   2> 4156 T453 oass.IndexSchema.readSchema default search field in schema is text
> [junit4:junit4]   2> 4160 T453 oass.IndexSchema.readSchema unique key field: id
> [junit4:junit4]   2> 4187 T453 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4]   2> 4194 T453 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4]   2> 4200 T453 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4]   2> 4202 T453 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4]   2> 4204 T453 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360019519403/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/control/data/
> [junit4:junit4]   2> 4209 T453 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5925c9a4
> [junit4:junit4]   2> 4209 T453 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
> [junit4:junit4]   2> 4211 T453 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/control/data forceNew:false
> [junit4:junit4]   2> 4211 T453 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/control/data
> [junit4:junit4]   2> 4212 T453 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/control/data/index/
> [junit4:junit4]   2> 4212 T453 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/control/data/index' doesn't exist. Creating new index...
> [junit4:junit4]   2> 4213 T453 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/control/data/index forceNew:false
> [junit4:junit4]   2> 4220 T453 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1713586c; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4]   2> 4220 T453 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
> [junit4:junit4]   2> 4221 T453 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/control/data/index
> [junit4:junit4]   2> 4222 T453 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
> [junit4:junit4]   2> 4226 T453 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4:junit4]   2> 4226 T453 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
> [junit4:junit4]   2> 4226 T453 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4:junit4]   2> 4227 T453 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
> [junit4:junit4]   2> 4228 T453 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4:junit4]   2> 4228 T453 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4:junit4]   2> 4229 T453 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4:junit4]   2> 4229 T453 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4:junit4]   2> 4230 T453 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4:junit4]   2> 4230 T453 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
> [junit4:junit4]   2> 4231 T453 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
> [junit4:junit4]   2> 4232 T453 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4:junit4]   2> 4232 T453 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
> [junit4:junit4]   2> 4233 T453 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
> [junit4:junit4]   2> 4233 T453 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
> [junit4:junit4]   2> 4234 T453 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4:junit4]   2> 4234 T453 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 4235 T453 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 4235 T453 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 4236 T453 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 4236 T453 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 4237 T453 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 4237 T453 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 4237 T453 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 4238 T453 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
> [junit4:junit4]   2> 4239 T453 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
> [junit4:junit4]   2> 4252 T453 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4:junit4]   2> 4257 T453 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
> [junit4:junit4]   2> 4258 T453 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
> [junit4:junit4]   2> 4261 T453 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/control/data
> [junit4:junit4]   2> 4263 T453 oass.SolrIndexSearcher.<init> Opening Searcher@78d4654e main
> [junit4:junit4]   2> 4289 T453 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4:junit4]   2> 4289 T453 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4:junit4]   2> 4290 T453 oashc.SpellCheckComponent.inform Initializing spell checkers
> [junit4:junit4]   2> 4309 T453 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
> [junit4:junit4]   2> 4362 T454 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@78d4654e main{StandardDirectoryReader(segments_1:1)}
> [junit4:junit4]   2> 4367 T453 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4]   2> 4368 T453 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4]   2> 4369 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720003 type:create cxid:0x50 zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
> [junit4:junit4]   2> 4654 T451 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 4655 T451 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
> [junit4:junit4]   2> 	  "operation":"state",
> [junit4:junit4]   2> 	  "numShards":"1",
> [junit4:junit4]   2> 	  "shard":null,
> [junit4:junit4]   2> 	  "roles":null,
> [junit4:junit4]   2> 	  "state":"down",
> [junit4:junit4]   2> 	  "core":"collection1",
> [junit4:junit4]   2> 	  "collection":"control_collection",
> [junit4:junit4]   2> 	  "node_name":"127.0.0.1:64828_sp_ffw%2Ff",
> [junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:64828/sp_ffw/f"}
> [junit4:junit4]   2> 4656 T451 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
> [junit4:junit4]   2> 4691 T451 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
> [junit4:junit4]   2> 4692 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720003 type:create cxid:0x58 zxid:0x2c txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
> [junit4:junit4]   2> 4729 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4:junit4]   2> 5382 T453 oasc.CoreContainer.registerCore registering core: collection1
> [junit4:junit4]   2> 5382 T453 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:64828/sp_ffw/f collection:control_collection shard:shard1
> [junit4:junit4]   2> 5393 T453 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
> [junit4:junit4]   2> 5410 T453 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
> [junit4:junit4]   2> 5432 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720003 type:delete cxid:0x72 zxid:0x36 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
> [junit4:junit4]   2> 5433 T453 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
> [junit4:junit4]   2> 5434 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720003 type:create cxid:0x73 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 5444 T453 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
> [junit4:junit4]   2> 5445 T453 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
> [junit4:junit4]   2> 5445 T453 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:64828/sp_ffw/f/collection1/
> [junit4:junit4]   2> 5446 T453 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
> [junit4:junit4]   2> 5446 T453 oasc.SyncStrategy.syncToMe http://127.0.0.1:64828/sp_ffw/f/collection1/ has no replicas
> [junit4:junit4]   2> 5446 T453 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:64828/sp_ffw/f/collection1/
> [junit4:junit4]   2> 5446 T453 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
> [junit4:junit4]   2> 5454 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720003 type:create cxid:0x7d zxid:0x3b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 6236 T451 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 6312 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4:junit4]   2> 6377 T453 oasc.ZkController.register We are http://127.0.0.1:64828/sp_ffw/f/collection1/ and leader is http://127.0.0.1:64828/sp_ffw/f/collection1/
> [junit4:junit4]   2> 6381 T453 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:64828/sp_ffw/f
> [junit4:junit4]   2> 6381 T453 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
> [junit4:junit4]   2> 6381 T453 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4:junit4]   2> 6382 T453 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4]   2> 6385 T453 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 6386 T428 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
> [junit4:junit4]   2> 6387 T428 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4:junit4]   2> 6388 T428 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4]   2> 6404 T428 oascsi.HttpClientUtil.createClient Creating new http client, config:
> [junit4:junit4]   2> 6421 T428 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45780/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@771cc1d1
> [junit4:junit4]   2> 6422 T428 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4]   2> 6423 T455 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45780. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4]   2> 6424 T455 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45780, initiating session
> [junit4:junit4]   2> 6424 T430 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:29639
> [junit4:junit4]   2> 6424 T430 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:29639
> [junit4:junit4]   2> 6426 T434 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca77aee720004 with negotiated timeout 10000 for client /140.211.11.196:29639
> [junit4:junit4]   2> 6426 T455 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45780, sessionid = 0x13ca77aee720004, negotiated timeout = 10000
> [junit4:junit4]   2> 6426 T456 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@771cc1d1 name:ZooKeeperConnection Watcher:127.0.0.1:45780/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4]   2> 6427 T428 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4]   2> 6428 T428 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
> [junit4:junit4]   2> 6433 T428 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
> [junit4:junit4]   2> 6622 T428 oejs.Server.doStart jetty-8.1.8.v20121106
> [junit4:junit4]   2> 6625 T428 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:58138
> [junit4:junit4]   2> 6626 T428 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4:junit4]   2> 6626 T428 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4:junit4]   2> 6627 T428 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360019523266
> [junit4:junit4]   2> 6627 T428 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360019523266/solr.xml
> [junit4:junit4]   2> 6628 T428 oasc.CoreContainer.<init> New CoreContainer 247900808
> [junit4:junit4]   2> 6628 T428 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360019523266/'
> [junit4:junit4]   2> 6629 T428 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360019523266/'
> [junit4:junit4]   2> 6689 T428 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
> [junit4:junit4]   2> 6690 T428 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4:junit4]   2> 6691 T428 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4:junit4]   2> 6691 T428 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4:junit4]   2> 6691 T428 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4:junit4]   2> 6692 T428 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4:junit4]   2> 6692 T428 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4:junit4]   2> 6693 T428 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4:junit4]   2> 6693 T428 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4:junit4]   2> 6694 T428 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
> [junit4:junit4]   2> 6730 T428 oasc.CoreContainer.load Registering Log Listener
> [junit4:junit4]   2> 6752 T428 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:45780/solr
> [junit4:junit4]   2> 6752 T428 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4:junit4]   2> 6753 T428 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45780 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@e24af2d
> [junit4:junit4]   2> 6754 T428 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4]   2> 6755 T466 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45780. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4]   2> 6756 T466 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45780, initiating session
> [junit4:junit4]   2> 6756 T430 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:33189
> [junit4:junit4]   2> 6756 T430 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:33189
> [junit4:junit4]   2> 6794 T434 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca77aee720005 with negotiated timeout 20000 for client /140.211.11.196:33189
> [junit4:junit4]   2> 6794 T466 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45780, sessionid = 0x13ca77aee720005, negotiated timeout = 20000
> [junit4:junit4]   2> 6795 T467 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e24af2d name:ZooKeeperConnection Watcher:127.0.0.1:45780 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4]   2> 6795 T428 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4]   2> 6797 T435 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca77aee720005
> [junit4:junit4]   2> 6798 T467 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4]   2> 6798 T428 oaz.ZooKeeper.close Session: 0x13ca77aee720005 closed
> [junit4:junit4]   2> 6799 T430 oazs.NIOServerCnxn.doIO WARNING caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x13ca77aee720005, likely client has closed socket
> [junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
> [junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
> [junit4:junit4]   2> 	at java.lang.Thread.run(Thread.java:722)
> [junit4:junit4]   2> 
> [junit4:junit4]   2> 6799 T428 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
> [junit4:junit4]   2> 6801 T430 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:33189 which had sessionid 0x13ca77aee720005
> [junit4:junit4]   2> 6816 T428 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45780/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@3151ee66
> [junit4:junit4]   2> 6818 T428 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4]   2> 6818 T468 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45780. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4]   2> 6819 T468 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45780, initiating session
> [junit4:junit4]   2> 6819 T430 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:15764
> [junit4:junit4]   2> 6820 T430 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:15764
> [junit4:junit4]   2> 6821 T434 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca77aee720006 with negotiated timeout 20000 for client /140.211.11.196:15764
> [junit4:junit4]   2> 6821 T468 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45780, sessionid = 0x13ca77aee720006, negotiated timeout = 20000
> [junit4:junit4]   2> 6822 T469 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3151ee66 name:ZooKeeperConnection Watcher:127.0.0.1:45780/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4]   2> 6822 T428 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4]   2> 6823 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720006 type:create cxid:0x1 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 6825 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720006 type:create cxid:0x2 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 6833 T428 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58138_sp_ffw%2Ff
> [junit4:junit4]   2> 6835 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720006 type:delete cxid:0x4 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:58138_sp_ffw%2Ff Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:58138_sp_ffw%2Ff
> [junit4:junit4]   2> 6836 T428 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:58138_sp_ffw%2Ff
> [junit4:junit4]   2> 6841 T450 oascc.ZkStateReader$3.process Updating live nodes... (2)
> [junit4:junit4]   2> 6841 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
> [junit4:junit4]   2> 6841 T456 oascc.ZkStateReader$3.process Updating live nodes... (2)
> [junit4:junit4]   2> 6851 T428 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
> [junit4:junit4]   2> 6859 T470 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360019523266/collection1
> [junit4:junit4]   2> 6859 T470 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
> [junit4:junit4]   2> 6860 T470 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4:junit4]   2> 6861 T470 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
> [junit4:junit4]   2> 6863 T470 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360019523266/collection1/'
> [junit4:junit4]   2> 6864 T470 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360019523266/collection1/lib/README' to classloader
> [junit4:junit4]   2> 6865 T470 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360019523266/collection1/lib/classes/' to classloader
> [junit4:junit4]   2> 6923 T470 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4:junit4]   2> 6999 T470 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4:junit4]   2> 7100 T470 oass.IndexSchema.readSchema Reading Solr Schema
> [junit4:junit4]   2> 7108 T470 oass.IndexSchema.readSchema Schema name=test
> [junit4:junit4]   2> 7759 T470 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4:junit4]   2> 7771 T470 oass.IndexSchema.readSchema default search field in schema is text
> [junit4:junit4]   2> 7776 T470 oass.IndexSchema.readSchema unique key field: id
> [junit4:junit4]   2> 7802 T470 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4]   2> 7808 T470 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4]   2> 7813 T470 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4]   2> 7815 T470 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4]   2> 7817 T470 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360019523266/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty1/
> [junit4:junit4]   2> 7817 T470 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5925c9a4
> [junit4:junit4]   2> 7818 T470 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
> [junit4:junit4]   2> 7819 T470 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty1 forceNew:false
> [junit4:junit4]   2> 7820 T470 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty1
> [junit4:junit4]   2> 7820 T470 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty1/index/
> [junit4:junit4]   2> 7820 T470 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty1/index' doesn't exist. Creating new index...
> [junit4:junit4]   2> 7822 T470 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty1/index forceNew:false
> [junit4:junit4]   2> 7828 T470 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3fb6de9f; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4]   2> 7829 T470 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
> [junit4:junit4]   2> 7830 T470 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty1/index
> [junit4:junit4]   2> 7830 T470 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
> [junit4:junit4]   2> 7831 T451 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 7832 T451 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
> [junit4:junit4]   2> 	  "operation":"state",
> [junit4:junit4]   2> 	  "numShards":"1",
> [junit4:junit4]   2> 	  "shard":"shard1",
> [junit4:junit4]   2> 	  "roles":null,
> [junit4:junit4]   2> 	  "state":"active",
> [junit4:junit4]   2> 	  "core":"collection1",
> [junit4:junit4]   2> 	  "collection":"control_collection",
> [junit4:junit4]   2> 	  "node_name":"127.0.0.1:64828_sp_ffw%2Ff",
> [junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:64828/sp_ffw/f"}
> [junit4:junit4]   2> 7832 T470 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4:junit4]   2> 7833 T470 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
> [junit4:junit4]   2> 7833 T470 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4:junit4]   2> 7834 T470 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
> [junit4:junit4]   2> 7834 T470 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4:junit4]   2> 7835 T470 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4:junit4]   2> 7835 T470 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4:junit4]   2> 7836 T470 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4:junit4]   2> 7836 T470 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4:junit4]   2> 7837 T470 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
> [junit4:junit4]   2> 7837 T470 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
> [junit4:junit4]   2> 7838 T470 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4:junit4]   2> 7838 T470 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
> [junit4:junit4]   2> 7839 T470 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
> [junit4:junit4]   2> 7839 T470 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
> [junit4:junit4]   2> 7840 T470 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4:junit4]   2> 7840 T470 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 7841 T470 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 7842 T470 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 7842 T470 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 7843 T470 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 7844 T470 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 7844 T470 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 7845 T470 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 7845 T469 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4]   2> 7845 T456 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4]   2> 7846 T470 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
> [junit4:junit4]   2> 7845 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4]   2> 7847 T470 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
> [junit4:junit4]   2> 7860 T470 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4:junit4]   2> 7865 T470 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
> [junit4:junit4]   2> 7866 T470 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
> [junit4:junit4]   2> 7869 T470 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty1
> [junit4:junit4]   2> 7871 T470 oass.SolrIndexSearcher.<init> Opening Searcher@4f74a09b main
> [junit4:junit4]   2> 7872 T470 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4:junit4]   2> 7872 T470 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4:junit4]   2> 7873 T470 oashc.SpellCheckComponent.inform Initializing spell checkers
> [junit4:junit4]   2> 7880 T470 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
> [junit4:junit4]   2> 7933 T471 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4f74a09b main{StandardDirectoryReader(segments_1:1)}
> [junit4:junit4]   2> 7938 T470 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4]   2> 7939 T470 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4]   2> 9358 T451 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 9359 T451 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4]   2> 	  "operation":"state",
> [junit4:junit4]   2> 	  "numShards":"2",
> [junit4:junit4]   2> 	  "shard":null,
> [junit4:junit4]   2> 	  "roles":null,
> [junit4:junit4]   2> 	  "state":"down",
> [junit4:junit4]   2> 	  "core":"collection1",
> [junit4:junit4]   2> 	  "collection":"collection1",
> [junit4:junit4]   2> 	  "node_name":"127.0.0.1:58138_sp_ffw%2Ff",
> [junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:58138/sp_ffw/f"}
> [junit4:junit4]   2> 9359 T451 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
> [junit4:junit4]   2> 9359 T451 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
> [junit4:junit4]   2> 9373 T456 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4]   2> 9375 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4]   2> 9375 T469 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4]   2> 9943 T470 oasc.CoreContainer.registerCore registering core: collection1
> [junit4:junit4]   2> 9943 T470 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:58138/sp_ffw/f collection:collection1 shard:shard1
> [junit4:junit4]   2> 9944 T470 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
> [junit4:junit4]   2> 9963 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720006 type:delete cxid:0x45 zxid:0x5b txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
> [junit4:junit4]   2> 9964 T470 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
> [junit4:junit4]   2> 9964 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720006 type:create cxid:0x46 zxid:0x5c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 9975 T470 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
> [junit4:junit4]   2> 9975 T470 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
> [junit4:junit4]   2> 9975 T470 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:58138/sp_ffw/f/collection1/
> [junit4:junit4]   2> 9975 T470 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
> [junit4:junit4]   2> 9976 T470 oasc.SyncStrategy.syncToMe http://127.0.0.1:58138/sp_ffw/f/collection1/ has no replicas
> [junit4:junit4]   2> 9976 T470 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:58138/sp_ffw/f/collection1/
> [junit4:junit4]   2> 9976 T470 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
> [junit4:junit4]   2> 9983 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720006 type:create cxid:0x50 zxid:0x60 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 10883 T451 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 10907 T469 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4]   2> 10907 T456 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4]   2> 10907 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4]   2> 10960 T470 oasc.ZkController.register We are http://127.0.0.1:58138/sp_ffw/f/collection1/ and leader is http://127.0.0.1:58138/sp_ffw/f/collection1/
> [junit4:junit4]   2> 10960 T470 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:58138/sp_ffw/f
> [junit4:junit4]   2> 10960 T470 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
> [junit4:junit4]   2> 10960 T470 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4:junit4]   2> 10961 T470 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4]   2> 10963 T470 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 10964 T428 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
> [junit4:junit4]   2> 10965 T428 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4:junit4]   2> 10966 T428 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4]   2> 11158 T428 oejs.Server.doStart jetty-8.1.8.v20121106
> [junit4:junit4]   2> 11162 T428 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:37419
> [junit4:junit4]   2> 11162 T428 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4:junit4]   2> 11163 T428 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4:junit4]   2> 11163 T428 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360019527813
> [junit4:junit4]   2> 11164 T428 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360019527813/solr.xml
> [junit4:junit4]   2> 11164 T428 oasc.CoreContainer.<init> New CoreContainer 157865851
> [junit4:junit4]   2> 11165 T428 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360019527813/'
> [junit4:junit4]   2> 11165 T428 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360019527813/'
> [junit4:junit4]   2> 11221 T428 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
> [junit4:junit4]   2> 11221 T428 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4:junit4]   2> 11222 T428 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4:junit4]   2> 11222 T428 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4:junit4]   2> 11223 T428 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4:junit4]   2> 11223 T428 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4:junit4]   2> 11224 T428 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4:junit4]   2> 11224 T428 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4:junit4]   2> 11224 T428 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4:junit4]   2> 11225 T428 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
> [junit4:junit4]   2> 11246 T428 oasc.CoreContainer.load Registering Log Listener
> [junit4:junit4]   2> 11266 T428 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:45780/solr
> [junit4:junit4]   2> 11267 T428 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4:junit4]   2> 11267 T428 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45780 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@5bd3155b
> [junit4:junit4]   2> 11269 T428 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4]   2> 11269 T481 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45780. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4]   2> 11270 T481 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45780, initiating session
> [junit4:junit4]   2> 11270 T430 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:40216
> [junit4:junit4]   2> 11270 T430 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:40216
> [junit4:junit4]   2> 11273 T434 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca77aee720007 with negotiated timeout 20000 for client /140.211.11.196:40216
> [junit4:junit4]   2> 11273 T481 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45780, sessionid = 0x13ca77aee720007, negotiated timeout = 20000
> [junit4:junit4]   2> 11273 T482 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5bd3155b name:ZooKeeperConnection Watcher:127.0.0.1:45780 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4]   2> 11273 T428 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4]   2> 11275 T435 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca77aee720007
> [junit4:junit4]   2> 11286 T482 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4]   2> 11286 T430 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:40216 which had sessionid 0x13ca77aee720007
> [junit4:junit4]   2> 11286 T428 oaz.ZooKeeper.close Session: 0x13ca77aee720007 closed
> [junit4:junit4]   2> 11287 T428 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
> [junit4:junit4]   2> 11298 T428 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45780/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@152e63d8
> [junit4:junit4]   2> 11299 T428 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4]   2> 11300 T483 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45780. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4]   2> 11300 T483 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45780, initiating session
> [junit4:junit4]   2> 11300 T430 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:54741
> [junit4:junit4]   2> 11301 T430 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:54741
> [junit4:junit4]   2> 11302 T434 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca77aee720008 with negotiated timeout 20000 for client /140.211.11.196:54741
> [junit4:junit4]   2> 11302 T483 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45780, sessionid = 0x13ca77aee720008, negotiated timeout = 20000
> [junit4:junit4]   2> 11303 T484 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@152e63d8 name:ZooKeeperConnection Watcher:127.0.0.1:45780/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4]   2> 11303 T428 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4]   2> 11304 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720008 type:create cxid:0x1 zxid:0x6d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 11306 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720008 type:create cxid:0x2 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 11313 T428 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37419_sp_ffw%2Ff
> [junit4:junit4]   2> 11314 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720008 type:delete cxid:0x4 zxid:0x6f txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:37419_sp_ffw%2Ff Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:37419_sp_ffw%2Ff
> [junit4:junit4]   2> 11315 T428 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:37419_sp_ffw%2Ff
> [junit4:junit4]   2> 11319 T469 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4:junit4]   2> 11320 T456 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4:junit4]   2> 11320 T469 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
> [junit4:junit4]   2> 11320 T456 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
> [junit4:junit4]   2> 11320 T450 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4:junit4]   2> 11320 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
> [junit4:junit4]   2> 11325 T428 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
> [junit4:junit4]   2> 11333 T485 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360019527813/collection1
> [junit4:junit4]   2> 11333 T485 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
> [junit4:junit4]   2> 11334 T485 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4:junit4]   2> 11334 T485 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
> [junit4:junit4]   2> 11336 T485 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360019527813/collection1/'
> [junit4:junit4]   2> 11338 T485 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360019527813/collection1/lib/README' to classloader
> [junit4:junit4]   2> 11338 T485 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360019527813/collection1/lib/classes/' to classloader
> [junit4:junit4]   2> 11393 T485 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4:junit4]   2> 11467 T485 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4:junit4]   2> 11568 T485 oass.IndexSchema.readSchema Reading Solr Schema
> [junit4:junit4]   2> 11576 T485 oass.IndexSchema.readSchema Schema name=test
> [junit4:junit4]   2> 12240 T485 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4:junit4]   2> 12252 T485 oass.IndexSchema.readSchema default search field in schema is text
> [junit4:junit4]   2> 12256 T485 oass.IndexSchema.readSchema unique key field: id
> [junit4:junit4]   2> 12283 T485 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4]   2> 12289 T485 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4]   2> 12294 T485 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4]   2> 12296 T485 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4]   2> 12298 T485 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360019527813/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty2/
> [junit4:junit4]   2> 12298 T485 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5925c9a4
> [junit4:junit4]   2> 12299 T485 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
> [junit4:junit4]   2> 12299 T485 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty2 forceNew:false
> [junit4:junit4]   2> 12300 T485 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty2
> [junit4:junit4]   2> 12300 T485 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty2/index/
> [junit4:junit4]   2> 12301 T485 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty2/index' doesn't exist. Creating new index...
> [junit4:junit4]   2> 12302 T485 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty2/index forceNew:false
> [junit4:junit4]   2> 12313 T485 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@475d785c; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4]   2> 12313 T485 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
> [junit4:junit4]   2> 12314 T485 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty2/index
> [junit4:junit4]   2> 12315 T485 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
> [junit4:junit4]   2> 12317 T485 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4:junit4]   2> 12317 T485 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
> [junit4:junit4]   2> 12317 T485 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4:junit4]   2> 12318 T485 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
> [junit4:junit4]   2> 12319 T485 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4:junit4]   2> 12319 T485 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4:junit4]   2> 12319 T485 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4:junit4]   2> 12320 T485 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4:junit4]   2> 12320 T485 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4:junit4]   2> 12321 T485 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
> [junit4:junit4]   2> 12322 T485 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
> [junit4:junit4]   2> 12322 T485 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4:junit4]   2> 12323 T485 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
> [junit4:junit4]   2> 12323 T485 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
> [junit4:junit4]   2> 12323 T485 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
> [junit4:junit4]   2> 12324 T485 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4:junit4]   2> 12324 T485 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 12325 T485 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 12326 T485 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 12326 T485 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 12327 T485 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 12327 T485 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 12328 T485 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 12328 T485 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 12329 T485 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
> [junit4:junit4]   2> 12330 T485 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
> [junit4:junit4]   2> 12342 T485 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4:junit4]   2> 12347 T485 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
> [junit4:junit4]   2> 12348 T485 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
> [junit4:junit4]   2> 12351 T485 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty2
> [junit4:junit4]   2> 12353 T485 oass.SolrIndexSearcher.<init> Opening Searcher@713d0cf main
> [junit4:junit4]   2> 12354 T485 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4:junit4]   2> 12354 T485 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4:junit4]   2> 12355 T485 oashc.SpellCheckComponent.inform Initializing spell checkers
> [junit4:junit4]   2> 12362 T485 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
> [junit4:junit4]   2> 12414 T486 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@713d0cf main{StandardDirectoryReader(segments_1:1)}
> [junit4:junit4]   2> 12420 T485 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4]   2> 12420 T485 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4]   2> 12423 T451 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 12424 T451 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4]   2> 	  "operation":"state",
> [junit4:junit4]   2> 	  "numShards":"2",
> [junit4:junit4]   2> 	  "shard":"shard1",
> [junit4:junit4]   2> 	  "roles":null,
> [junit4:junit4]   2> 	  "state":"active",
> [junit4:junit4]   2> 	  "core":"collection1",
> [junit4:junit4]   2> 	  "collection":"collection1",
> [junit4:junit4]   2> 	  "node_name":"127.0.0.1:58138_sp_ffw%2Ff",
> [junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:58138/sp_ffw/f"}
> [junit4:junit4]   2> 12435 T451 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4]   2> 	  "operation":"state",
> [junit4:junit4]   2> 	  "numShards":"2",
> [junit4:junit4]   2> 	  "shard":null,
> [junit4:junit4]   2> 	  "roles":null,
> [junit4:junit4]   2> 	  "state":"down",
> [junit4:junit4]   2> 	  "core":"collection1",
> [junit4:junit4]   2> 	  "collection":"collection1",
> [junit4:junit4]   2> 	  "node_name":"127.0.0.1:37419_sp_ffw%2Ff",
> [junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:37419/sp_ffw/f"}
> [junit4:junit4]   2> 12435 T451 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
> [junit4:junit4]   2> 12435 T451 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
> [junit4:junit4]   2> 12447 T456 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4]   2> 12447 T484 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4]   2> 12447 T469 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4]   2> 12447 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4]   2> 13423 T485 oasc.CoreContainer.registerCore registering core: collection1
> [junit4:junit4]   2> 13423 T485 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:37419/sp_ffw/f collection:collection1 shard:shard2
> [junit4:junit4]   2> 13424 T485 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
> [junit4:junit4]   2> 13469 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720008 type:delete cxid:0x42 zxid:0x7d txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
> [junit4:junit4]   2> 13470 T485 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
> [junit4:junit4]   2> 13471 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720008 type:create cxid:0x43 zxid:0x7e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 13481 T485 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
> [junit4:junit4]   2> 13481 T485 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
> [junit4:junit4]   2> 13481 T485 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:37419/sp_ffw/f/collection1/
> [junit4:junit4]   2> 13481 T485 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
> [junit4:junit4]   2> 13482 T485 oasc.SyncStrategy.syncToMe http://127.0.0.1:37419/sp_ffw/f/collection1/ has no replicas
> [junit4:junit4]   2> 13482 T485 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:37419/sp_ffw/f/collection1/
> [junit4:junit4]   2> 13482 T485 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
> [junit4:junit4]   2> 13487 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720008 type:create cxid:0x4c zxid:0x81 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 13958 T451 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 13978 T456 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4]   2> 13978 T469 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4]   2> 13978 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4]   2> 13978 T484 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4]   2> 14000 T485 oasc.ZkController.register We are http://127.0.0.1:37419/sp_ffw/f/collection1/ and leader is http://127.0.0.1:37419/sp_ffw/f/collection1/
> [junit4:junit4]   2> 14000 T485 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:37419/sp_ffw/f
> [junit4:junit4]   2> 14000 T485 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
> [junit4:junit4]   2> 14000 T485 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4:junit4]   2> 14001 T485 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4]   2> 14003 T485 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 14004 T428 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
> [junit4:junit4]   2> 14005 T428 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4:junit4]   2> 14005 T428 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4]   2> 14191 T428 oejs.Server.doStart jetty-8.1.8.v20121106
> [junit4:junit4]   2> 14194 T428 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:30255
> [junit4:junit4]   2> 14195 T428 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4:junit4]   2> 14195 T428 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4:junit4]   2> 14196 T428 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360019530853
> [junit4:junit4]   2> 14196 T428 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360019530853/solr.xml
> [junit4:junit4]   2> 14197 T428 oasc.CoreContainer.<init> New CoreContainer 363940238
> [junit4:junit4]   2> 14197 T428 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360019530853/'
> [junit4:junit4]   2> 14198 T428 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360019530853/'
> [junit4:junit4]   2> 14253 T428 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
> [junit4:junit4]   2> 14253 T428 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4:junit4]   2> 14254 T428 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4:junit4]   2> 14254 T428 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4:junit4]   2> 14254 T428 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4:junit4]   2> 14255 T428 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4:junit4]   2> 14255 T428 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4:junit4]   2> 14256 T428 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4:junit4]   2> 14256 T428 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4:junit4]   2> 14257 T428 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
> [junit4:junit4]   2> 14280 T428 oasc.CoreContainer.load Registering Log Listener
> [junit4:junit4]   2> 14300 T428 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:45780/solr
> [junit4:junit4]   2> 14301 T428 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4:junit4]   2> 14301 T428 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45780 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@5c65b07d
> [junit4:junit4]   2> 14302 T428 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4]   2> 14303 T496 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45780. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4]   2> 14304 T496 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45780, initiating session
> [junit4:junit4]   2> 14304 T430 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:12017
> [junit4:junit4]   2> 14305 T430 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:12017
> [junit4:junit4]   2> 14307 T434 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca77aee720009 with negotiated timeout 20000 for client /140.211.11.196:12017
> [junit4:junit4]   2> 14307 T496 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45780, sessionid = 0x13ca77aee720009, negotiated timeout = 20000
> [junit4:junit4]   2> 14307 T497 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5c65b07d name:ZooKeeperConnection Watcher:127.0.0.1:45780 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4]   2> 14307 T428 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4]   2> 14309 T435 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca77aee720009
> [junit4:junit4]   2> 14322 T497 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4]   2> 14322 T430 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:12017 which had sessionid 0x13ca77aee720009
> [junit4:junit4]   2> 14322 T428 oaz.ZooKeeper.close Session: 0x13ca77aee720009 closed
> [junit4:junit4]   2> 14323 T428 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
> [junit4:junit4]   2> 14337 T428 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45780/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@21b9d8f4
> [junit4:junit4]   2> 14338 T428 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4]   2> 14338 T498 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45780. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4]   2> 14338 T498 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45780, initiating session
> [junit4:junit4]   2> 14339 T430 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:15905
> [junit4:junit4]   2> 14339 T430 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:15905
> [junit4:junit4]   2> 14340 T434 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca77aee72000a with negotiated timeout 20000 for client /140.211.11.196:15905
> [junit4:junit4]   2> 14340 T498 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45780, sessionid = 0x13ca77aee72000a, negotiated timeout = 20000
> [junit4:junit4]   2> 14341 T499 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@21b9d8f4 name:ZooKeeperConnection Watcher:127.0.0.1:45780/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4]   2> 14341 T428 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4]   2> 14342 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee72000a type:create cxid:0x1 zxid:0x8e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 14344 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee72000a type:create cxid:0x2 zxid:0x8f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 14351 T428 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:30255_sp_ffw%2Ff
> [junit4:junit4]   2> 14352 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee72000a type:delete cxid:0x4 zxid:0x90 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:30255_sp_ffw%2Ff Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:30255_sp_ffw%2Ff
> [junit4:junit4]   2> 14353 T428 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:30255_sp_ffw%2Ff
> [junit4:junit4]   2> 14356 T484 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
> [junit4:junit4]   2> 14357 T456 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4:junit4]   2> 14357 T469 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4:junit4]   2> 14357 T456 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4]   2> 14357 T450 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4:junit4]   2> 14358 T469 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4]   2> 14358 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4]   2> 14359 T484 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4:junit4]   2> 14363 T428 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
> [junit4:junit4]   2> 14371 T500 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360019530853/collection1
> [junit4:junit4]   2> 14371 T500 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
> [junit4:junit4]   2> 14372 T500 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4:junit4]   2> 14372 T500 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
> [junit4:junit4]   2> 14374 T500 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360019530853/collection1/'
> [junit4:junit4]   2> 14376 T500 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360019530853/collection1/lib/README' to classloader
> [junit4:junit4]   2> 14376 T500 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360019530853/collection1/lib/classes/' to classloader
> [junit4:junit4]   2> 14431 T500 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4:junit4]   2> 14506 T500 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4:junit4]   2> 14607 T500 oass.IndexSchema.readSchema Reading Solr Schema
> [junit4:junit4]   2> 14615 T500 oass.IndexSchema.readSchema Schema name=test
> [junit4:junit4]   2> 15272 T500 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4:junit4]   2> 15285 T500 oass.IndexSchema.readSchema default search field in schema is text
> [junit4:junit4]   2> 15289 T500 oass.IndexSchema.readSchema unique key field: id
> [junit4:junit4]   2> 15315 T500 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4]   2> 15321 T500 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4]   2> 15326 T500 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4]   2> 15328 T500 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4]   2> 15330 T500 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360019530853/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3/
> [junit4:junit4]   2> 15330 T500 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5925c9a4
> [junit4:junit4]   2> 15331 T500 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
> [junit4:junit4]   2> 15331 T500 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3 forceNew:false
> [junit4:junit4]   2> 15332 T500 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3
> [junit4:junit4]   2> 15332 T500 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3/index/
> [junit4:junit4]   2> 15333 T500 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3/index' doesn't exist. Creating new index...
> [junit4:junit4]   2> 15334 T500 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3/index forceNew:false
> [junit4:junit4]   2> 15341 T500 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@57b3153c; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4]   2> 15341 T500 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
> [junit4:junit4]   2> 15342 T500 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3/index
> [junit4:junit4]   2> 15342 T500 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
> [junit4:junit4]   2> 15345 T500 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4:junit4]   2> 15345 T500 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
> [junit4:junit4]   2> 15345 T500 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4:junit4]   2> 15346 T500 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
> [junit4:junit4]   2> 15346 T500 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4:junit4]   2> 15347 T500 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4:junit4]   2> 15347 T500 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4:junit4]   2> 15348 T500 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4:junit4]   2> 15348 T500 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4:junit4]   2> 15349 T500 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
> [junit4:junit4]   2> 15349 T500 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
> [junit4:junit4]   2> 15350 T500 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4:junit4]   2> 15351 T500 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
> [junit4:junit4]   2> 15351 T500 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
> [junit4:junit4]   2> 15351 T500 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
> [junit4:junit4]   2> 15352 T500 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4:junit4]   2> 15352 T500 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 15353 T500 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 15353 T500 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 15354 T500 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 15355 T500 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 15355 T500 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 15356 T500 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 15356 T500 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 15357 T500 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
> [junit4:junit4]   2> 15357 T500 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
> [junit4:junit4]   2> 15370 T500 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4:junit4]   2> 15375 T500 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
> [junit4:junit4]   2> 15376 T500 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
> [junit4:junit4]   2> 15379 T500 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3
> [junit4:junit4]   2> 15381 T500 oass.SolrIndexSearcher.<init> Opening Searcher@58c9d231 main
> [junit4:junit4]   2> 15382 T500 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4:junit4]   2> 15382 T500 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4:junit4]   2> 15383 T500 oashc.SpellCheckComponent.inform Initializing spell checkers
> [junit4:junit4]   2> 15390 T500 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
> [junit4:junit4]   2> 15446 T501 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@58c9d231 main{StandardDirectoryReader(segments_1:1)}
> [junit4:junit4]   2> 15452 T500 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4]   2> 15452 T500 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4]   2> 15492 T451 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 15494 T451 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4]   2> 	  "operation":"state",
> [junit4:junit4]   2> 	  "numShards":"2",
> [junit4:junit4]   2> 	  "shard":"shard2",
> [junit4:junit4]   2> 	  "roles":null,
> [junit4:junit4]   2> 	  "state":"active",
> [junit4:junit4]   2> 	  "core":"collection1",
> [junit4:junit4]   2> 	  "collection":"collection1",
> [junit4:junit4]   2> 	  "node_name":"127.0.0.1:37419_sp_ffw%2Ff",
> [junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:37419/sp_ffw/f"}
> [junit4:junit4]   2> 15499 T451 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4]   2> 	  "operation":"state",
> [junit4:junit4]   2> 	  "numShards":"2",
> [junit4:junit4]   2> 	  "shard":null,
> [junit4:junit4]   2> 	  "roles":null,
> [junit4:junit4]   2> 	  "state":"down",
> [junit4:junit4]   2> 	  "core":"collection1",
> [junit4:junit4]   2> 	  "collection":"collection1",
> [junit4:junit4]   2> 	  "node_name":"127.0.0.1:30255_sp_ffw%2Ff",
> [junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:30255/sp_ffw/f"}
> [junit4:junit4]   2> 15499 T451 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
> [junit4:junit4]   2> 15500 T451 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
> [junit4:junit4]   2> 15518 T456 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4]   2> 15518 T469 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4]   2> 15518 T484 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4]   2> 15518 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4]   2> 15518 T499 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4]   2> 16455 T500 oasc.CoreContainer.registerCore registering core: collection1
> [junit4:junit4]   2> 16455 T500 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:30255/sp_ffw/f collection:collection1 shard:shard1
> [junit4:junit4]   2> 16486 T500 oasc.ZkController.register We are http://127.0.0.1:30255/sp_ffw/f/collection1/ and leader is http://127.0.0.1:58138/sp_ffw/f/collection1/
> [junit4:junit4]   2> 16486 T500 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:30255/sp_ffw/f
> [junit4:junit4]   2> 16486 T500 oasc.ZkController.checkRecovery Core needs to recover:collection1
> [junit4:junit4]   2> 16487 T500 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
> [junit4:junit4]   2> ASYNC  NEW_CORE C14 name=collection1 org.apache.solr.core.SolrCore@3e2cc207 url=http://127.0.0.1:30255/sp_ffw/f/collection1 node=127.0.0.1:30255_sp_ffw%2Ff C14_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:30255_sp_ffw%2Ff, base_url=http://127.0.0.1:30255/sp_ffw/f}
> [junit4:junit4]   2> 16494 T502 C14 P30255 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
> [junit4:junit4]   2> 16494 T500 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 16495 T502 C14 P30255 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
> [junit4:junit4]   2> 16495 T502 C14 P30255 oasc.ZkController.publish publishing core=collection1 state=recovering
> [junit4:junit4]   2> 16495 T502 C14 P30255 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4]   2> 16496 T428 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
> [junit4:junit4]   2> 16496 T428 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4:junit4]   2> 16497 T502 C14 P30255 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4]   2> 16497 T428 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4]   2> 16694 T428 oejs.Server.doStart jetty-8.1.8.v20121106
> [junit4:junit4]   2> 16697 T428 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:55162
> [junit4:junit4]   2> 16698 T428 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4:junit4]   2> 16698 T428 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4:junit4]   2> 16699 T428 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360019533349
> [junit4:junit4]   2> 16699 T428 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360019533349/solr.xml
> [junit4:junit4]   2> 16700 T428 oasc.CoreContainer.<init> New CoreContainer 63645884
> [junit4:junit4]   2> 16700 T428 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360019533349/'
> [junit4:junit4]   2> 16701 T428 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360019533349/'
> [junit4:junit4]   2> 16769 T428 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
> [junit4:junit4]   2> 16770 T428 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4:junit4]   2> 16770 T428 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4:junit4]   2> 16771 T428 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4:junit4]   2> 16771 T428 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4:junit4]   2> 16772 T428 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4:junit4]   2> 16772 T428 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4:junit4]   2> 16773 T428 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4:junit4]   2> 16773 T428 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4:junit4]   2> 16774 T428 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
> [junit4:junit4]   2> 16796 T428 oasc.CoreContainer.load Registering Log Listener
> [junit4:junit4]   2> 16816 T428 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:45780/solr
> [junit4:junit4]   2> 16817 T428 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4:junit4]   2> 16817 T428 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45780 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@3d5bc62e
> [junit4:junit4]   2> 16818 T428 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4]   2> 16819 T512 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45780. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4]   2> 16820 T512 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45780, initiating session
> [junit4:junit4]   2> 16820 T430 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:47360
> [junit4:junit4]   2> 16821 T430 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:47360
> [junit4:junit4]   2> 16855 T434 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca77aee72000b with negotiated timeout 20000 for client /140.211.11.196:47360
> [junit4:junit4]   2> 16856 T512 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45780, sessionid = 0x13ca77aee72000b, negotiated timeout = 20000
> [junit4:junit4]   2> 16856 T513 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3d5bc62e name:ZooKeeperConnection Watcher:127.0.0.1:45780 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4]   2> 16856 T428 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4]   2> 16858 T435 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca77aee72000b
> [junit4:junit4]   2> 16871 T513 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4]   2> 16871 T430 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:47360 which had sessionid 0x13ca77aee72000b
> [junit4:junit4]   2> 16871 T428 oaz.ZooKeeper.close Session: 0x13ca77aee72000b closed
> [junit4:junit4]   2> 16872 T428 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
> [junit4:junit4]   2> 16883 T428 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45780/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@3155ba4
> [junit4:junit4]   2> 16884 T428 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4]   2> 16884 T514 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45780. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4]   2> 16885 T514 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45780, initiating session
> [junit4:junit4]   2> 16885 T430 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:49058
> [junit4:junit4]   2> 16886 T430 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:49058
> [junit4:junit4]   2> 16887 T434 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca77aee72000c with negotiated timeout 20000 for client /140.211.11.196:49058
> [junit4:junit4]   2> 16887 T514 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45780, sessionid = 0x13ca77aee72000c, negotiated timeout = 20000
> [junit4:junit4]   2> 16887 T515 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3155ba4 name:ZooKeeperConnection Watcher:127.0.0.1:45780/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4]   2> 16888 T428 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4]   2> 16889 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee72000c type:create cxid:0x1 zxid:0xa0 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 16890 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee72000c type:create cxid:0x2 zxid:0xa1 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 16898 T428 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55162_sp_ffw%2Ff
> [junit4:junit4]   2> 16899 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee72000c type:delete cxid:0x4 zxid:0xa2 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:55162_sp_ffw%2Ff Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:55162_sp_ffw%2Ff
> [junit4:junit4]   2> 16900 T428 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:55162_sp_ffw%2Ff
> [junit4:junit4]   2> 16904 T484 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4]   2> 16905 T456 oascc.ZkStateReader$3.process Updating live nodes... (5)
> [junit4:junit4]   2> 16905 T450 oascc.ZkStateReader$3.process Updating live nodes... (5)
> [junit4:junit4]   2> 16905 T456 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 16905 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 16905 T499 oascc.ZkStateReader$3.process Updating live nodes... (5)
> [junit4:junit4]   2> 16905 T469 oascc.ZkStateReader$3.process Updating live nodes... (5)
> [junit4:junit4]   2> 16906 T499 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 16906 T469 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 16908 T484 oascc.ZkStateReader$3.process Updating live nodes... (5)
> [junit4:junit4]   2> 16912 T428 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
> [junit4:junit4]   2> 16920 T516 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360019533349/collection1
> [junit4:junit4]   2> 16921 T516 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
> [junit4:junit4]   2> 16922 T516 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4:junit4]   2> 16922 T516 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
> [junit4:junit4]   2> 16924 T516 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360019533349/collection1/'
> [junit4:junit4]   2> 16925 T516 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360019533349/collection1/lib/README' to classloader
> [junit4:junit4]   2> 16926 T516 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360019533349/collection1/lib/classes/' to classloader
> [junit4:junit4]   2> 16981 T516 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4:junit4]   2> 17059 T516 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4:junit4]   2> 17062 T451 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 17063 T451 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4]   2> 	  "operation":"state",
> [junit4:junit4]   2> 	  "numShards":"2",
> [junit4:junit4]   2> 	  "shard":"shard1",
> [junit4:junit4]   2> 	  "roles":null,
> [junit4:junit4]   2> 	  "state":"recovering",
> [junit4:junit4]   2> 	  "core":"collection1",
> [junit4:junit4]   2> 	  "collection":"collection1",
> [junit4:junit4]   2> 	  "node_name":"127.0.0.1:30255_sp_ffw%2Ff",
> [junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:30255/sp_ffw/f"}
> [junit4:junit4]   2> 17077 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 17077 T499 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 17077 T484 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 17077 T469 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 17077 T456 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 17160 T516 oass.IndexSchema.readSchema Reading Solr Schema
> [junit4:junit4]   2> 17160 T515 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 17168 T516 oass.IndexSchema.readSchema Schema name=test
> [junit4:junit4]   2> 17833 T516 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4:junit4]   2> 17848 T516 oass.IndexSchema.readSchema default search field in schema is text
> [junit4:junit4]   2> 17852 T516 oass.IndexSchema.readSchema unique key field: id
> [junit4:junit4]   2> 17889 T516 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4]   2> 17895 T516 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4]   2> 17902 T516 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4]   2> 17904 T516 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4]   2> 17906 T516 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360019533349/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty4/
> [junit4:junit4]   2> 17906 T516 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5925c9a4
> [junit4:junit4]   2> 17907 T516 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
> [junit4:junit4]   2> 17908 T516 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty4 forceNew:false
> [junit4:junit4]   2> 17909 T516 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty4
> [junit4:junit4]   2> 17910 T516 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty4/index/
> [junit4:junit4]   2> 17911 T516 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty4/index' doesn't exist. Creating new index...
> [junit4:junit4]   2> 17913 T516 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty4/index forceNew:false
> [junit4:junit4]   2> 17931 T516 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2092ab58; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4]   2> 17931 T516 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
> [junit4:junit4]   2> 17932 T516 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty4/index
> [junit4:junit4]   2> 17933 T516 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
> [junit4:junit4]   2> 17935 T516 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4:junit4]   2> 17935 T516 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
> [junit4:junit4]   2> 17936 T516 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4:junit4]   2> 17936 T516 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
> [junit4:junit4]   2> 17937 T516 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4:junit4]   2> 17937 T516 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4:junit4]   2> 17937 T516 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4:junit4]   2> 17938 T516 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4:junit4]   2> 17939 T516 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4:junit4]   2> 17939 T516 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
> [junit4:junit4]   2> 17940 T516 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
> [junit4:junit4]   2> 17940 T516 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4:junit4]   2> 17941 T516 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
> [junit4:junit4]   2> 17941 T516 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
> [junit4:junit4]   2> 17941 T516 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
> [junit4:junit4]   2> 17942 T516 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4:junit4]   2> 17943 T516 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 17943 T516 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 17944 T516 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 17944 T516 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 17945 T516 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 17945 T516 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 17946 T516 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 17947 T516 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 17947 T516 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
> [junit4:junit4]   2> 17948 T516 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
> [junit4:junit4]   2> 17969 T516 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4:junit4]   2> 17976 T516 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
> [junit4:junit4]   2> 17978 T516 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
> [junit4:junit4]   2> 17982 T516 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty4
> [junit4:junit4]   2> 17985 T516 oass.SolrIndexSearcher.<init> Opening Searcher@2b5bd32c main
> [junit4:junit4]   2> 17986 T516 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4:junit4]   2> 17986 T516 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4:junit4]   2> 17987 T516 oashc.SpellCheckComponent.inform Initializing spell checkers
> [junit4:junit4]   2> 17995 T516 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
> [junit4:junit4]   2> 18048 T517 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2b5bd32c main{StandardDirectoryReader(segments_1:1)}
> [junit4:junit4]   2> 18054 T516 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4]   2> 18054 T516 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4]   2> 18585 T451 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 18587 T451 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4]   2> 	  "operation":"state",
> [junit4:junit4]   2> 	  "numShards":"2",
> [junit4:junit4]   2> 	  "shard":null,
> [junit4:junit4]   2> 	  "roles":null,
> [junit4:junit4]   2> 	  "state":"down",
> [junit4:junit4]   2> 	  "core":"collection1",
> [junit4:junit4]   2> 	  "collection":"collection1",
> [junit4:junit4]   2> 	  "node_name":"127.0.0.1:55162_sp_ffw%2Ff",
> [junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:55162/sp_ffw/f"}
> [junit4:junit4]   2> 18587 T451 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
> [junit4:junit4]   2> 18587 T451 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
> [junit4:junit4]   2> 18602 T456 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 18602 T469 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 18602 T484 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 18602 T499 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 18602 T515 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 18602 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 19057 T516 oasc.CoreContainer.registerCore registering core: collection1
> [junit4:junit4]   2> 19057 T516 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:55162/sp_ffw/f collection:collection1 shard:shard2
> [junit4:junit4]   2> 19062 T516 oasc.ZkController.register We are http://127.0.0.1:55162/sp_ffw/f/collection1/ and leader is http://127.0.0.1:37419/sp_ffw/f/collection1/
> [junit4:junit4]   2> 19062 T516 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:55162/sp_ffw/f
> [junit4:junit4]   2> 19062 T516 oasc.ZkController.checkRecovery Core needs to recover:collection1
> [junit4:junit4]   2> 19062 T516 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
> [junit4:junit4]   2> ASYNC  NEW_CORE C15 name=collection1 org.apache.solr.core.SolrCore@41d5df31 url=http://127.0.0.1:55162/sp_ffw/f/collection1 node=127.0.0.1:55162_sp_ffw%2Ff C15_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:55162_sp_ffw%2Ff, base_url=http://127.0.0.1:55162/sp_ffw/f}
> [junit4:junit4]   2> 19063 T518 C15 P55162 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
> [junit4:junit4]   2> 19063 T518 C15 P55162 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
> [junit4:junit4]   2> 19064 T516 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 19064 T518 C15 P55162 oasc.ZkController.publish publishing core=collection1 state=recovering
> [junit4:junit4]   2> 19064 T518 C15 P55162 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4]   2> 19065 T428 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
> [junit4:junit4]   2> 19066 T518 C15 P55162 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4]   2> 19066 T428 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4:junit4]   2> 19066 T428 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4]   2> 19085 T428 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 19087 T428 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
> [junit4:junit4]   2> 19088 T428 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2>  C14_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:30255_sp_ffw%2Ff, base_url=http://127.0.0.1:30255/sp_ffw/f}
> [junit4:junit4]   2> 19539 T502 C14 P30255 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:58138/sp_ffw/f/collection1/ core=collection1 - recoveringAfterStartup=true
> [junit4:junit4]   2> 19548 T502 C14 P30255 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&connTimeout=30000&socketTimeout=30000&retry=false
> [junit4:junit4]   2> 19566 T502 C14 P30255 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:30255/sp_ffw/f START replicas=[http://127.0.0.1:58138/sp_ffw/f/collection1/] nUpdates=100
> [junit4:junit4]   2> 19567 T502 C14 P30255 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
> [junit4:junit4]   2> 19568 T502 C14 P30255 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
> [junit4:junit4]   2> 19568 T502 C14 P30255 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
> [junit4:junit4]   2> 19568 T502 C14 P30255 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
> [junit4:junit4]   2> 19569 T502 C14 P30255 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
> [junit4:junit4]   2> 19569 T502 C14 P30255 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:58138/sp_ffw/f/collection1/. core=collection1
> [junit4:junit4]   2> 19569 T502 C14 P30255 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4]   2> ASYNC  NEW_CORE C16 name=collection1 org.apache.solr.core.SolrCore@60a5494 url=http://127.0.0.1:58138/sp_ffw/f/collection1 node=127.0.0.1:58138_sp_ffw%2Ff C16_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:58138_sp_ffw%2Ff, base_url=http://127.0.0.1:58138/sp_ffw/f, leader=true}
> [junit4:junit4]   2> 19580 T460 C16 P58138 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1 
> [junit4:junit4]   2> 19599 T461 C16 P58138 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
> [junit4:junit4]   2> 19600 T461 C16 P58138 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty1
> [junit4:junit4]   2> 19604 T461 C16 P58138 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3fb6de9f; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4]   2> 19605 T461 C16 P58138 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
> [junit4:junit4]   2> 19635 T461 C16 P58138 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3fb6de9f; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3fb6de9f; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
> [junit4:junit4]   2> 19635 T461 C16 P58138 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
> [junit4:junit4]   2> 19637 T461 C16 P58138 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty1
> [junit4:junit4]   2> 19637 T461 C16 P58138 oass.SolrIndexSearcher.<init> Opening Searcher@7765731d realtime
> [junit4:junit4]   2> 19638 T461 C16 P58138 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4]   2> 19638 T461 C16 P58138 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 56
> [junit4:junit4]   2> 19696 T502 C14 P30255 oash.ReplicationHandler.inform Commits will be reserved for  10000
> [junit4:junit4]   2> 19697 T502 C14 P30255 oascsi.HttpClientUtil.createClient Creating new http client, config:connTimeout=5000&socketTimeout=20000&allowCompression=false&maxConnections=10000&maxConnectionsPerHost=10000
> [junit4:junit4]   2> 19723 T502 C14 P30255 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
> [junit4:junit4]   2> 19726 T462 C16 P58138 oash.ReplicationHandler.inform Commits will be reserved for  10000
> [junit4:junit4]   2> 19726 T462 C16 P58138 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2 
> [junit4:junit4]   2> 19727 T502 C14 P30255 oash.SnapPuller.fetchLatestIndex Master's generation: 2
> [junit4:junit4]   2> 19728 T502 C14 P30255 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
> [junit4:junit4]   2> 19728 T502 C14 P30255 oash.SnapPuller.fetchLatestIndex Starting replication process
> [junit4:junit4]   2> 19731 T462 C16 P58138 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty1
> [junit4:junit4]   2> 19732 T462 C16 P58138 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty1/index
> [junit4:junit4]   2> 19732 T462 C16 P58138 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=2 
> [junit4:junit4]   2> 19733 T502 C14 P30255 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
> [junit4:junit4]   2> 19734 T502 C14 P30255 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3/index.20130205001216565 forceNew:false
> [junit4:junit4]   2> 19735 T502 C14 P30255 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3
> [junit4:junit4]   2> 19735 T502 C14 P30255 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3/index.20130205001216565 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6ffa2f97; maxCacheMB=48.0 maxMergeSizeMB=4.0)) fullCopy=true
> [junit4:junit4]   2> 19749 T462 C16 P58138 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=5 
> [junit4:junit4]   2> 19762 T502 C14 P30255 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3/index
> [junit4:junit4]   2> 19763 T502 C14 P30255 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
> [junit4:junit4]   2> 19764 T502 C14 P30255 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130205001216565
> [junit4:junit4]   2> 19771 T502 C14 P30255 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3
> [junit4:junit4]   2> 19773 T502 C14 P30255 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3
> [junit4:junit4]   2> 19773 T502 C14 P30255 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3/index/ new=./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3/index.20130205001216565
> [junit4:junit4]   2> 19777 T502 C14 P30255 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3/index.20130205001216565 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6ffa2f97; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
> [junit4:junit4]   2> 19777 T502 C14 P30255 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
> [junit4:junit4]   2> 19779 T502 C14 P30255 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3
> [junit4:junit4]   2> 19783 T502 C14 P30255 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
> [junit4:junit4]   2> 19784 T502 C14 P30255 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
> [junit4:junit4]   2> 19784 T502 C14 P30255 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
> [junit4:junit4]   2> 19785 T502 C14 P30255 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3/index.20130205001216565
> [junit4:junit4]   2> 19786 T502 C14 P30255 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3
> [junit4:junit4]   2> 19789 T502 C14 P30255 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3/index.20130205001216565 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6ffa2f97; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
> [junit4:junit4]   2> 19790 T502 C14 P30255 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
> [junit4:junit4]   2> 19791 T502 C14 P30255 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
> [junit4:junit4]   2> 19791 T502 C14 P30255 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3
> [junit4:junit4]   2> 19792 T502 C14 P30255 oass.SolrIndexSearcher.<init> Opening Searcher@2dd212cf main
> [junit4:junit4]   2> 19793 T501 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2dd212cf main{StandardDirectoryReader(segments_2:1:nrt)}
> [junit4:junit4]   2> 19794 T501 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3/index
> [junit4:junit4]   2> 19795 T502 C14 P30255 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
> [junit4:junit4]   2> 19798 T502 C14 P30255 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3/index.20130205001216565 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6ffa2f97; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3/index.20130205001216565 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6ffa2f97; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_3,generation=3,filenames=[segments_3]
> [junit4:junit4]   2> 19799 T502 C14 P30255 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
> [junit4:junit4]   2> 19800 T502 C14 P30255 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3
> [junit4:junit4]   2> 19800 T502 C14 P30255 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4]   2> 19800 T502 C14 P30255 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3/index.20130205001216565
> [junit4:junit4]   2> 19801 T502 C14 P30255 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3/index
> [junit4:junit4]   2> 19801 T502 C14 P30255 oasc.RecoveryStrategy.replay No replay needed. core=collection1
> [junit4:junit4]   2> 19801 T502 C14 P30255 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
> [junit4:junit4]   2> 19802 T502 C14 P30255 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4:junit4]   2> 19802 T502 C14 P30255 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4]   2> 19804 T502 C14 P30255 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
> [junit4:junit4]   2> 20092 T428 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 20111 T451 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 20112 T451 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4]   2> 	  "operation":"state",
> [junit4:junit4]   2> 	  "numShards":"2",
> [junit4:junit4]   2> 	  "shard":"shard2",
> [junit4:junit4]   2> 	  "roles":null,
> [junit4:junit4]   2> 	  "state":"recovering",
> [junit4:junit4]   2> 	  "core":"collection1",
> [junit4:junit4]   2> 	  "collection":"collection1",
> [junit4:junit4]   2> 	  "node_name":"127.0.0.1:55162_sp_ffw%2Ff",
> [junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:55162/sp_ffw/f"}
> [junit4:junit4]   2> 20118 T451 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4]   2> 	  "operation":"state",
> [junit4:junit4]   2> 	  "numShards":"2",
> [junit4:junit4]   2> 	  "shard":"shard1",
> [junit4:junit4]   2> 	  "roles":null,
> [junit4:junit4]   2> 	  "state":"active",
> [junit4:junit4]   2> 	  "core":"collection1",
> [junit4:junit4]   2> 	  "collection":"collection1",
> [junit4:junit4]   2> 	  "node_name":"127.0.0.1:30255_sp_ffw%2Ff",
> [junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:30255/sp_ffw/f"}
> [junit4:junit4]   2> 20130 T456 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 20131 T515 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 20131 T484 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 20131 T499 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 20131 T469 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 20130 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 21095 T428 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 22098 T428 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2>  C15_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:55162_sp_ffw%2Ff, base_url=http://127.0.0.1:55162/sp_ffw/f}
> [junit4:junit4]   2> 23089 T518 C15 P55162 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:37419/sp_ffw/f/collection1/ core=collection1 - recoveringAfterStartup=true
> [junit4:junit4]   2> 23089 T518 C15 P55162 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:55162/sp_ffw/f START replicas=[http://127.0.0.1:37419/sp_ffw/f/collection1/] nUpdates=100
> [junit4:junit4]   2> 23090 T518 C15 P55162 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
> [junit4:junit4]   2> 23090 T518 C15 P55162 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
> [junit4:junit4]   2> 23090 T518 C15 P55162 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
> [junit4:junit4]   2> 23091 T518 C15 P55162 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
> [junit4:junit4]   2> 23091 T518 C15 P55162 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
> [junit4:junit4]   2> 23091 T518 C15 P55162 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:37419/sp_ffw/f/collection1/. core=collection1
> [junit4:junit4]   2> 23092 T518 C15 P55162 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4]   2> 23101 T428 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> ASYNC  NEW_CORE C17 name=collection1 org.apache.solr.core.SolrCore@ab64f2 url=http:
> 
> [...truncated too long message...]
> 
> .io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:111)
> [junit4:junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:264)
> [junit4:junit4]    > 	at org.apache.http.impl.conn.DefaultResponseParser.parseHead(DefaultResponseParser.java:98)
> [junit4:junit4]    > 	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:252)
> [junit4:junit4]    > 	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:282)
> [junit4:junit4]    > 	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:247)
> [junit4:junit4]    > 	at org.apache.http.impl.conn.AbstractClientConnAdapter.receiveResponseHeader(AbstractClientConnAdapter.java:216)
> [junit4:junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:298)
> [junit4:junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
> [junit4:junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:647)
> [junit4:junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:464)
> [junit4:junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
> [junit4:junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
> [junit4:junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:732)
> [junit4:junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:353)
> [junit4:junit4]    > 	... 5 more
> [junit4:junit4]    > Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=939, name=testExecutor-345-thread-5, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
> [junit4:junit4]    > Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:30255/sp_ffw/f
> [junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([715CCD8973136CF0]:0)
> [junit4:junit4]    > 	at org.apache.solr.cloud.BasicDistributedZkTest$1.run(BasicDistributedZkTest.java:658)
> [junit4:junit4]    > 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> [junit4:junit4]    > 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> [junit4:junit4]    > 	at java.lang.Thread.run(Thread.java:722)
> [junit4:junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:30255/sp_ffw/f
> [junit4:junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:412)
> [junit4:junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181)
> [junit4:junit4]    > 	at org.apache.solr.cloud.BasicDistributedZkTest$1.run(BasicDistributedZkTest.java:656)
> [junit4:junit4]    > 	... 3 more
> [junit4:junit4]    > Caused by: java.net.SocketTimeoutException: Read timed out
> [junit4:junit4]    > 	at java.net.SocketInputStream.socketRead0(Native Method)
> [junit4:junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:150)
> [junit4:junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:121)
> [junit4:junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:149)
> [junit4:junit4]    > 	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:111)
> [junit4:junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:264)
> [junit4:junit4]    > 	at org.apache.http.impl.conn.DefaultResponseParser.parseHead(DefaultResponseParser.java:98)
> [junit4:junit4]    > 	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:252)
> [junit4:junit4]    > 	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:282)
> [junit4:junit4]    > 	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:247)
> [junit4:junit4]    > 	at org.apache.http.impl.conn.AbstractClientConnAdapter.receiveResponseHeader(AbstractClientConnAdapter.java:216)
> [junit4:junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:298)
> [junit4:junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
> [junit4:junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:647)
> [junit4:junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:464)
> [junit4:junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
> [junit4:junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
> [junit4:junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:732)
> [junit4:junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:353)
> [junit4:junit4]    > 	... 5 more
> [junit4:junit4]   2> 388062 T428 oas.SolrTestCaseJ4.deleteCore ###deleteCore
> [junit4:junit4]   2> 388169 T431 oazs.SessionTrackerImpl.run SessionTrackerImpl exited loop!
> [junit4:junit4]   2> 388258 T499 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
> [junit4:junit4]   2> 388258 T499 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
> [junit4:junit4]   2> 388258 T499 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4]   2> 388430 T515 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
> [junit4:junit4]   2> 388430 T515 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
> [junit4:junit4]   2> 388430 T515 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
> [junit4:junit4]   2> 388430 T515 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4]   2> NOTE: test params are: codec=Lucene42: {n_f1=PostingsFormat(name=NestedPulsing), foo_b=PostingsFormat(name=Memory doPackFST= true), cat=Lucene41(blocksize=128), foo_d=Pulsing41(freqCutoff=13 minBlockSize=38 maxBlockSize=83), foo_f=PostingsFormat(name=Memory doPackFST= true), n_tl1=PostingsFormat(name=NestedPulsing), n_d1=Lucene41(blocksize=128), rnd_b=Pulsing41(freqCutoff=13 minBlockSize=38 maxBlockSize=83), intDefault=Lucene41(blocksize=128), n_td1=PostingsFormat(name=NestedPulsing), timestamp=Lucene41(blocksize=128), id=Pulsing41(freqCutoff=13 minBlockSize=38 maxBlockSize=83), range_facet_sl=PostingsFormat(name=Memory doPackFST= true), range_facet_si=PostingsFormat(name=NestedPulsing), oddField_s=PostingsFormat(name=Memory doPackFST= true), sequence_i=PostingsFormat(name=Memory doPackFST= true), name=Pulsing41(freqCutoff=13 minBlockSize=38 maxBlockSize=83), foo_i=PostingsFormat(name=NestedPulsing), regex_dup_B_s=Lucene41(blocksize=128), multiDefault=PostingsFormat(name=NestedPulsing), n_tf1=Lucene41(blocksize=128), n_dt1=PostingsFormat(name=NestedPulsing), genre_s=PostingsFormat(name=Memory doPackFST= true), author_t=PostingsFormat(name=NestedPulsing), n_ti1=Pulsing41(freqCutoff=13 minBlockSize=38 maxBlockSize=83), range_facet_l=Lucene41(blocksize=128), text=PostingsFormat(name=Memory doPackFST= true), _version_=Lucene41(blocksize=128), val_i=Pulsing41(freqCutoff=13 minBlockSize=38 maxBlockSize=83), SubjectTerms_mfacet=Lucene41(blocksize=128), series_t=PostingsFormat(name=NestedPulsing), a_t=Lucene41(blocksize=128), n_tdt1=Lucene41(blocksize=128), regex_dup_A_s=PostingsFormat(name=Memory doPackFST= true), price=PostingsFormat(name=Memory doPackFST= true), other_tl1=Lucene41(blocksize=128), n_l1=Lucene41(blocksize=128), a_si=PostingsFormat(name=NestedPulsing), inStock=PostingsFormat(name=Memory doPackFST= true)}, sim=DefaultSimilarity, locale=et, timezone=ECT
> [junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=2,free=327414984,total=475004928
> [junit4:junit4]   2> NOTE: All tests run in this JVM: [RequestHandlersTest, TestFiltering, TestRandomFaceting, TestDefaultSimilarityFactory, BinaryUpdateRequestHandlerTest, DistributedSpellCheckComponentTest, TestGroupingSearch, JsonLoaderTest, BasicDistributedZkTest]
> [junit4:junit4] Completed on J0 in 388.65s, 1 test, 1 error <<< FAILURES!
> 
> [...truncated 582 lines...]
> BUILD FAILED
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:346: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:39: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build.xml:178: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/common-build.xml:428: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:1176: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:841: There were test failures: 249 suites, 1038 tests, 1 error, 23 ignored (5 assumptions)
> 
> Total time: 39 minutes 26 seconds
> Build step 'Invoke Ant' marked build as failure
> Archiving artifacts
> Recording test results
> Email was triggered for: Failure
> Sending email for trigger: Failure
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[JENKINS] Lucene-Solr-Tests-trunk-java7 - Build # 3708 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-java7/3708/

1 tests failed.
FAILED:  org.apache.solr.cloud.BasicDistributedZkTest.testDistribSearch

Error Message:
Captured an uncaught exception in thread: Thread[id=935, name=testExecutor-345-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=935, name=testExecutor-345-thread-1, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:30255/sp_ffw/f
	at __randomizedtesting.SeedInfo.seed([715CCD8973136CF0]:0)
	at org.apache.solr.cloud.BasicDistributedZkTest$1.run(BasicDistributedZkTest.java:658)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
	at java.lang.Thread.run(Thread.java:722)
Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:30255/sp_ffw/f
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:412)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181)
	at org.apache.solr.cloud.BasicDistributedZkTest$1.run(BasicDistributedZkTest.java:656)
	... 3 more
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:150)
	at java.net.SocketInputStream.read(SocketInputStream.java:121)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:149)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:111)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:264)
	at org.apache.http.impl.conn.DefaultResponseParser.parseHead(DefaultResponseParser.java:98)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:252)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:282)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:247)
	at org.apache.http.impl.conn.AbstractClientConnAdapter.receiveResponseHeader(AbstractClientConnAdapter.java:216)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:298)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:647)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:464)
	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:732)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:353)
	... 5 more




Build Log:
[...truncated 8817 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
[junit4:junit4]   2> 0 T428 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /sp_ffw/f
[junit4:junit4]   2> 4 T428 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZkTest-1360019516837
[junit4:junit4]   2> 11 T428 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 15 T429 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 73 T429 oaz.Environment.logEnv Server environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT
[junit4:junit4]   2> 74 T429 oaz.Environment.logEnv Server environment:host.name=lucene.zones.apache.org
[junit4:junit4]   2> 74 T429 oaz.Environment.logEnv Server environment:java.version=1.7.0_09
[junit4:junit4]   2> 74 T429 oaz.Environment.logEnv Server environment:java.vendor=Oracle Corporation
[junit4:junit4]   2> 74 T429 oaz.Environment.logEnv Server environment:java.home=/usr/local/openjdk7/jre
[junit4:junit4]   2> 74 T429 oaz.Environment.logEnv Server environment:java.class.path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/classes/test:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/codecs/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-solrj/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/analysis/common/lucene-analyzers-common-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/highlighter/lucene-highlighter-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/memory/lucene-memory-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/misc/lucene-misc-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/spatial/lucene-spatial-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/suggest/lucene-suggest-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/grouping/lucene-grouping-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/queries/lucene-queries-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/queryparser/lucene-queryparser-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/cglib-nodep-2.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/commons-cli-1.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/commons-codec-1.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/commons-fileupload-1.2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/commons-lang-2.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/easymock-3.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/guava-13.0.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/javax.servlet-api-3.0.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/objenesis-1.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/spatial4j-0.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/commons-io-2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/httpclient-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/httpcore-4.1.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/httpmime-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/jcl-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/slf4j-api-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/slf4j-jdk14-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/wstx-asl-3.2.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/zookeeper-3.4.5.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-continuation-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-deploy-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-http-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-io-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-jmx-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-security-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-server-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-servlet-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-util-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-webapp-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-xml-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/servlet-api-3.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/example-DIH/solr/db/lib/derby-10.9.1.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/test-framework/lib/junit-4.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/test-framework/lib/randomizedtesting-runner-2.0.8.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/home/hudson/.ant/lib/ivy-2.2.0.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/usr/local/openjdk7/lib/tools.jar:/home/hudson/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.0.8.jar
[junit4:junit4]   2> 75 T429 oaz.Environment.logEnv Server environment:java.library.path=/usr/local/openjdk7/jre/lib/amd64/server:/usr/local/openjdk7/jre/lib/amd64:/usr/local/openjdk7/jre/../lib/amd64:/usr/local/openjdk6/jre/lib/amd64/server:/usr/local/openjdk6/jre/lib/amd64:/usr/local/openjdk6/jre/../lib/amd64:/usr/java/packages/lib/amd64:/lib:/usr/lib:/usr/local/lib
[junit4:junit4]   2> 75 T429 oaz.Environment.logEnv Server environment:java.io.tmpdir=.
[junit4:junit4]   2> 75 T429 oaz.Environment.logEnv Server environment:java.compiler=<NA>
[junit4:junit4]   2> 75 T429 oaz.Environment.logEnv Server environment:os.name=FreeBSD
[junit4:junit4]   2> 75 T429 oaz.Environment.logEnv Server environment:os.arch=amd64
[junit4:junit4]   2> 76 T429 oaz.Environment.logEnv Server environment:os.version=9.0-RELEASE
[junit4:junit4]   2> 76 T429 oaz.Environment.logEnv Server environment:user.name=hudson
[junit4:junit4]   2> 76 T429 oaz.Environment.logEnv Server environment:user.home=/home/hudson
[junit4:junit4]   2> 76 T429 oaz.Environment.logEnv Server environment:user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 109 T429 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 109 T429 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4]   2> 109 T429 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4]   2> 115 T429 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 116 T428 oasc.ZkTestServer.run start zk server on port:45780
[junit4:junit4]   2> 179 T428 oaz.Environment.logEnv Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT
[junit4:junit4]   2> 180 T428 oaz.Environment.logEnv Client environment:host.name=lucene.zones.apache.org
[junit4:junit4]   2> 180 T428 oaz.Environment.logEnv Client environment:java.version=1.7.0_09
[junit4:junit4]   2> 180 T428 oaz.Environment.logEnv Client environment:java.vendor=Oracle Corporation
[junit4:junit4]   2> 180 T428 oaz.Environment.logEnv Client environment:java.home=/usr/local/openjdk7/jre
[junit4:junit4]   2> 181 T428 oaz.Environment.logEnv Client environment:java.class.path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/classes/test:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/codecs/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-solrj/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/analysis/common/lucene-analyzers-common-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/highlighter/lucene-highlighter-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/memory/lucene-memory-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/misc/lucene-misc-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/spatial/lucene-spatial-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/suggest/lucene-suggest-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/grouping/lucene-grouping-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/queries/lucene-queries-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/queryparser/lucene-queryparser-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/cglib-nodep-2.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/commons-cli-1.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/commons-codec-1.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/commons-fileupload-1.2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/commons-lang-2.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/easymock-3.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/guava-13.0.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/javax.servlet-api-3.0.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/objenesis-1.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/spatial4j-0.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/commons-io-2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/httpclient-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/httpcore-4.1.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/httpmime-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/jcl-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/slf4j-api-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/slf4j-jdk14-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/wstx-asl-3.2.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/zookeeper-3.4.5.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-continuation-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-deploy-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-http-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-io-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-jmx-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-security-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-server-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-servlet-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-util-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-webapp-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-xml-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/servlet-api-3.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/example-DIH/solr/db/lib/derby-10.9.1.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/test-framework/lib/junit-4.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/test-framework/lib/randomizedtesting-runner-2.0.8.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/home/hudson/.ant/lib/ivy-2.2.0.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/usr/local/openjdk7/lib/tools.jar:/home/hudson/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.0.8.jar
[junit4:junit4]   2> 182 T428 oaz.Environment.logEnv Client environment:java.library.path=/usr/local/openjdk7/jre/lib/amd64/server:/usr/local/openjdk7/jre/lib/amd64:/usr/local/openjdk7/jre/../lib/amd64:/usr/local/openjdk6/jre/lib/amd64/server:/usr/local/openjdk6/jre/lib/amd64:/usr/local/openjdk6/jre/../lib/amd64:/usr/java/packages/lib/amd64:/lib:/usr/lib:/usr/local/lib
[junit4:junit4]   2> 182 T428 oaz.Environment.logEnv Client environment:java.io.tmpdir=.
[junit4:junit4]   2> 182 T428 oaz.Environment.logEnv Client environment:java.compiler=<NA>
[junit4:junit4]   2> 183 T429 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4]   2> 183 T428 oaz.Environment.logEnv Client environment:os.name=FreeBSD
[junit4:junit4]   2> 183 T428 oaz.Environment.logEnv Client environment:os.arch=amd64
[junit4:junit4]   2> 183 T428 oaz.Environment.logEnv Client environment:os.version=9.0-RELEASE
[junit4:junit4]   2> 183 T428 oaz.Environment.logEnv Client environment:user.name=hudson
[junit4:junit4]   2> 184 T428 oaz.Environment.logEnv Client environment:user.home=/home/hudson
[junit4:junit4]   2> 184 T428 oaz.Environment.logEnv Client environment:user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 187 T428 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45780 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@b2da32d
[junit4:junit4]   2> 246 T428 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 263 T432 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45780. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 266 T432 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45780, initiating session
[junit4:junit4]   2> 267 T430 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:21112
[junit4:junit4]   2> 284 T430 oazs.NIOServerCnxn.doIO WARNING Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running
[junit4:junit4]   2> 285 T430 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:21112 (no session established for client)
[junit4:junit4]   2> 285 T432 oaz.ClientCnxn$SendThread.run Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
[junit4:junit4]   2> 2163 T432 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45780. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 2164 T432 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45780, initiating session
[junit4:junit4]   2> 2164 T430 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:19269
[junit4:junit4]   2> 2178 T430 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:19269
[junit4:junit4]   2> 2194 T434 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 2235 T434 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca77aee720000 with negotiated timeout 10000 for client /140.211.11.196:19269
[junit4:junit4]   2> 2236 T432 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45780, sessionid = 0x13ca77aee720000, negotiated timeout = 10000
[junit4:junit4]   2> 2240 T433 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b2da32d name:ZooKeeperConnection Watcher:127.0.0.1:45780 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2240 T428 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 2245 T428 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 2329 T435 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca77aee720000
[junit4:junit4]   2> 2331 T433 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 2331 T428 oaz.ZooKeeper.close Session: 0x13ca77aee720000 closed
[junit4:junit4]   2> 2331 T430 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:19269 which had sessionid 0x13ca77aee720000
[junit4:junit4]   2> 2331 T428 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45780/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@79ba1ec8
[junit4:junit4]   2> 2332 T428 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 2333 T436 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45780. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 2333 T436 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45780, initiating session
[junit4:junit4]   2> 2334 T430 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:49718
[junit4:junit4]   2> 2334 T430 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:49718
[junit4:junit4]   2> 2336 T434 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca77aee720001 with negotiated timeout 10000 for client /140.211.11.196:49718
[junit4:junit4]   2> 2336 T436 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45780, sessionid = 0x13ca77aee720001, negotiated timeout = 10000
[junit4:junit4]   2> 2337 T437 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@79ba1ec8 name:ZooKeeperConnection Watcher:127.0.0.1:45780/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2337 T428 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 2343 T428 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 2349 T428 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 2360 T428 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 2364 T428 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 2369 T428 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 2397 T428 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 2413 T428 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 2414 T428 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 2521 T428 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 2522 T428 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 2528 T428 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 2529 T428 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 2535 T428 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 2535 T428 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 2542 T428 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 2542 T428 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 2549 T428 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 2549 T428 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 2555 T428 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 2556 T428 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 2562 T428 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 2563 T428 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 2569 T435 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca77aee720001
[junit4:junit4]   2> 2570 T437 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 2570 T428 oaz.ZooKeeper.close Session: 0x13ca77aee720001 closed
[junit4:junit4]   2> 2570 T430 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:49718 which had sessionid 0x13ca77aee720001
[junit4:junit4]   2> 2769 T428 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 2773 T428 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:64828
[junit4:junit4]   2> 2774 T428 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 2774 T428 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 2775 T428 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360019519403
[junit4:junit4]   2> 2775 T428 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360019519403/solr.xml
[junit4:junit4]   2> 2776 T428 oasc.CoreContainer.<init> New CoreContainer 1882182574
[junit4:junit4]   2> 2777 T428 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360019519403/'
[junit4:junit4]   2> 2777 T428 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360019519403/'
[junit4:junit4]   2> 2839 T428 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 2839 T428 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 2840 T428 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 2840 T428 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 2841 T428 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 2841 T428 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 2842 T428 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 2842 T428 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 2843 T428 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 2843 T428 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 2871 T428 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 2895 T428 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:45780/solr
[junit4:junit4]   2> 2921 T428 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 2922 T428 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45780 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@31415815
[junit4:junit4]   2> 2923 T428 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 2923 T447 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45780. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 2924 T447 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45780, initiating session
[junit4:junit4]   2> 2924 T430 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:24071
[junit4:junit4]   2> 2925 T430 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:24071
[junit4:junit4]   2> 2926 T434 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca77aee720002 with negotiated timeout 20000 for client /140.211.11.196:24071
[junit4:junit4]   2> 2926 T447 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45780, sessionid = 0x13ca77aee720002, negotiated timeout = 20000
[junit4:junit4]   2> 2927 T448 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@31415815 name:ZooKeeperConnection Watcher:127.0.0.1:45780 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2927 T428 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 2929 T435 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca77aee720002
[junit4:junit4]   2> 2930 T448 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 2930 T430 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:24071 which had sessionid 0x13ca77aee720002
[junit4:junit4]   2> 2930 T428 oaz.ZooKeeper.close Session: 0x13ca77aee720002 closed
[junit4:junit4]   2> 2936 T428 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 2955 T428 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45780/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@56fe2145
[junit4:junit4]   2> 2956 T428 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 2957 T449 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45780. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 2957 T449 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45780, initiating session
[junit4:junit4]   2> 2958 T430 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:49673
[junit4:junit4]   2> 2958 T430 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:49673
[junit4:junit4]   2> 2960 T434 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca77aee720003 with negotiated timeout 20000 for client /140.211.11.196:49673
[junit4:junit4]   2> 2960 T449 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45780, sessionid = 0x13ca77aee720003, negotiated timeout = 20000
[junit4:junit4]   2> 2960 T450 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@56fe2145 name:ZooKeeperConnection Watcher:127.0.0.1:45780/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 2960 T428 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 2982 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2994 T428 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 2998 T428 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:64828_sp_ffw%2Ff
[junit4:junit4]   2> 3004 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720003 type:delete cxid:0x7 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:64828_sp_ffw%2Ff Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:64828_sp_ffw%2Ff
[junit4:junit4]   2> 3006 T428 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:64828_sp_ffw%2Ff
[junit4:junit4]   2> 3012 T428 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 3047 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720003 type:delete cxid:0x15 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 3048 T428 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 3053 T428 oasc.Overseer.start Overseer (id=89130239068864515-127.0.0.1:64828_sp_ffw%2Ff-n_0000000000) starting
[junit4:junit4]   2> 3055 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720003 type:create cxid:0x1a zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3086 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720003 type:create cxid:0x1b zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3088 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720003 type:create cxid:0x1c zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3102 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720003 type:create cxid:0x1d zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3104 T452 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 3115 T428 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 3119 T428 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 3144 T451 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 3161 T453 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360019519403/collection1
[junit4:junit4]   2> 3161 T453 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 3162 T453 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 3162 T453 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 3165 T453 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360019519403/collection1/'
[junit4:junit4]   2> 3166 T453 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360019519403/collection1/lib/README' to classloader
[junit4:junit4]   2> 3167 T453 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360019519403/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 3246 T453 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 3326 T453 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 3427 T453 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 3435 T453 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 4143 T453 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 4156 T453 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 4160 T453 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 4187 T453 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 4194 T453 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 4200 T453 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 4202 T453 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 4204 T453 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1360019519403/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/control/data/
[junit4:junit4]   2> 4209 T453 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5925c9a4
[junit4:junit4]   2> 4209 T453 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 4211 T453 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/control/data forceNew:false
[junit4:junit4]   2> 4211 T453 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/control/data
[junit4:junit4]   2> 4212 T453 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/control/data/index/
[junit4:junit4]   2> 4212 T453 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 4213 T453 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/control/data/index forceNew:false
[junit4:junit4]   2> 4220 T453 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1713586c; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 4220 T453 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 4221 T453 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/control/data/index
[junit4:junit4]   2> 4222 T453 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 4226 T453 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 4226 T453 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 4226 T453 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 4227 T453 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 4228 T453 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 4228 T453 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 4229 T453 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 4229 T453 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 4230 T453 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 4230 T453 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 4231 T453 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 4232 T453 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 4232 T453 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 4233 T453 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 4233 T453 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 4234 T453 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 4234 T453 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4235 T453 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4235 T453 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4236 T453 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4236 T453 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4237 T453 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4237 T453 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4237 T453 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4238 T453 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 4239 T453 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 4252 T453 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 4257 T453 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 4258 T453 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 4261 T453 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/control/data
[junit4:junit4]   2> 4263 T453 oass.SolrIndexSearcher.<init> Opening Searcher@78d4654e main
[junit4:junit4]   2> 4289 T453 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 4289 T453 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 4290 T453 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 4309 T453 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 4362 T454 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@78d4654e main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 4367 T453 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 4368 T453 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 4369 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720003 type:create cxid:0x50 zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 4654 T451 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 4655 T451 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:64828_sp_ffw%2Ff",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:64828/sp_ffw/f"}
[junit4:junit4]   2> 4656 T451 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4]   2> 4691 T451 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 4692 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720003 type:create cxid:0x58 zxid:0x2c txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 4729 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 5382 T453 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 5382 T453 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:64828/sp_ffw/f collection:control_collection shard:shard1
[junit4:junit4]   2> 5393 T453 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 5410 T453 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4]   2> 5432 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720003 type:delete cxid:0x72 zxid:0x36 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4]   2> 5433 T453 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 5434 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720003 type:create cxid:0x73 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5444 T453 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 5445 T453 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 5445 T453 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:64828/sp_ffw/f/collection1/
[junit4:junit4]   2> 5446 T453 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 5446 T453 oasc.SyncStrategy.syncToMe http://127.0.0.1:64828/sp_ffw/f/collection1/ has no replicas
[junit4:junit4]   2> 5446 T453 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:64828/sp_ffw/f/collection1/
[junit4:junit4]   2> 5446 T453 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4]   2> 5454 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720003 type:create cxid:0x7d zxid:0x3b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6236 T451 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6312 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 6377 T453 oasc.ZkController.register We are http://127.0.0.1:64828/sp_ffw/f/collection1/ and leader is http://127.0.0.1:64828/sp_ffw/f/collection1/
[junit4:junit4]   2> 6381 T453 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:64828/sp_ffw/f
[junit4:junit4]   2> 6381 T453 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 6381 T453 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 6382 T453 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 6385 T453 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6386 T428 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 6387 T428 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 6388 T428 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 6404 T428 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 6421 T428 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45780/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@771cc1d1
[junit4:junit4]   2> 6422 T428 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 6423 T455 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45780. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 6424 T455 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45780, initiating session
[junit4:junit4]   2> 6424 T430 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:29639
[junit4:junit4]   2> 6424 T430 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:29639
[junit4:junit4]   2> 6426 T434 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca77aee720004 with negotiated timeout 10000 for client /140.211.11.196:29639
[junit4:junit4]   2> 6426 T455 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45780, sessionid = 0x13ca77aee720004, negotiated timeout = 10000
[junit4:junit4]   2> 6426 T456 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@771cc1d1 name:ZooKeeperConnection Watcher:127.0.0.1:45780/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 6427 T428 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 6428 T428 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 6433 T428 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 6622 T428 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 6625 T428 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:58138
[junit4:junit4]   2> 6626 T428 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 6626 T428 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 6627 T428 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360019523266
[junit4:junit4]   2> 6627 T428 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360019523266/solr.xml
[junit4:junit4]   2> 6628 T428 oasc.CoreContainer.<init> New CoreContainer 247900808
[junit4:junit4]   2> 6628 T428 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360019523266/'
[junit4:junit4]   2> 6629 T428 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360019523266/'
[junit4:junit4]   2> 6689 T428 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 6690 T428 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 6691 T428 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 6691 T428 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 6691 T428 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 6692 T428 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 6692 T428 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 6693 T428 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 6693 T428 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 6694 T428 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 6730 T428 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 6752 T428 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:45780/solr
[junit4:junit4]   2> 6752 T428 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 6753 T428 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45780 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@e24af2d
[junit4:junit4]   2> 6754 T428 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 6755 T466 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45780. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 6756 T466 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45780, initiating session
[junit4:junit4]   2> 6756 T430 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:33189
[junit4:junit4]   2> 6756 T430 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:33189
[junit4:junit4]   2> 6794 T434 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca77aee720005 with negotiated timeout 20000 for client /140.211.11.196:33189
[junit4:junit4]   2> 6794 T466 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45780, sessionid = 0x13ca77aee720005, negotiated timeout = 20000
[junit4:junit4]   2> 6795 T467 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e24af2d name:ZooKeeperConnection Watcher:127.0.0.1:45780 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 6795 T428 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 6797 T435 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca77aee720005
[junit4:junit4]   2> 6798 T467 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 6798 T428 oaz.ZooKeeper.close Session: 0x13ca77aee720005 closed
[junit4:junit4]   2> 6799 T430 oazs.NIOServerCnxn.doIO WARNING caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x13ca77aee720005, likely client has closed socket
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4:junit4]   2> 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 
[junit4:junit4]   2> 6799 T428 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 6801 T430 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:33189 which had sessionid 0x13ca77aee720005
[junit4:junit4]   2> 6816 T428 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45780/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@3151ee66
[junit4:junit4]   2> 6818 T428 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 6818 T468 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45780. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 6819 T468 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45780, initiating session
[junit4:junit4]   2> 6819 T430 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:15764
[junit4:junit4]   2> 6820 T430 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:15764
[junit4:junit4]   2> 6821 T434 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca77aee720006 with negotiated timeout 20000 for client /140.211.11.196:15764
[junit4:junit4]   2> 6821 T468 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45780, sessionid = 0x13ca77aee720006, negotiated timeout = 20000
[junit4:junit4]   2> 6822 T469 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3151ee66 name:ZooKeeperConnection Watcher:127.0.0.1:45780/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 6822 T428 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 6823 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720006 type:create cxid:0x1 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6825 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720006 type:create cxid:0x2 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6833 T428 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58138_sp_ffw%2Ff
[junit4:junit4]   2> 6835 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720006 type:delete cxid:0x4 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:58138_sp_ffw%2Ff Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:58138_sp_ffw%2Ff
[junit4:junit4]   2> 6836 T428 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:58138_sp_ffw%2Ff
[junit4:junit4]   2> 6841 T450 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 6841 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6841 T456 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 6851 T428 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 6859 T470 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360019523266/collection1
[junit4:junit4]   2> 6859 T470 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 6860 T470 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 6861 T470 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 6863 T470 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360019523266/collection1/'
[junit4:junit4]   2> 6864 T470 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360019523266/collection1/lib/README' to classloader
[junit4:junit4]   2> 6865 T470 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360019523266/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 6923 T470 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 6999 T470 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 7100 T470 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 7108 T470 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 7759 T470 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 7771 T470 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 7776 T470 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 7802 T470 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 7808 T470 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 7813 T470 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 7815 T470 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 7817 T470 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1360019523266/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty1/
[junit4:junit4]   2> 7817 T470 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5925c9a4
[junit4:junit4]   2> 7818 T470 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 7819 T470 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty1 forceNew:false
[junit4:junit4]   2> 7820 T470 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty1
[junit4:junit4]   2> 7820 T470 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty1/index/
[junit4:junit4]   2> 7820 T470 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 7822 T470 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty1/index forceNew:false
[junit4:junit4]   2> 7828 T470 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3fb6de9f; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 7829 T470 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 7830 T470 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty1/index
[junit4:junit4]   2> 7830 T470 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 7831 T451 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 7832 T451 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:64828_sp_ffw%2Ff",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:64828/sp_ffw/f"}
[junit4:junit4]   2> 7832 T470 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 7833 T470 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 7833 T470 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 7834 T470 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 7834 T470 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 7835 T470 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 7835 T470 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 7836 T470 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 7836 T470 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 7837 T470 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 7837 T470 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 7838 T470 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 7838 T470 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 7839 T470 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 7839 T470 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 7840 T470 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 7840 T470 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7841 T470 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7842 T470 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7842 T470 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7843 T470 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7844 T470 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7844 T470 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7845 T470 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7845 T469 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7845 T456 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7846 T470 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 7845 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7847 T470 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 7860 T470 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 7865 T470 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 7866 T470 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 7869 T470 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty1
[junit4:junit4]   2> 7871 T470 oass.SolrIndexSearcher.<init> Opening Searcher@4f74a09b main
[junit4:junit4]   2> 7872 T470 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 7872 T470 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 7873 T470 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 7880 T470 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 7933 T471 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4f74a09b main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 7938 T470 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 7939 T470 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 9358 T451 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9359 T451 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:58138_sp_ffw%2Ff",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:58138/sp_ffw/f"}
[junit4:junit4]   2> 9359 T451 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4]   2> 9359 T451 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 9373 T456 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 9375 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 9375 T469 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 9943 T470 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 9943 T470 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:58138/sp_ffw/f collection:collection1 shard:shard1
[junit4:junit4]   2> 9944 T470 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 9963 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720006 type:delete cxid:0x45 zxid:0x5b txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 9964 T470 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 9964 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720006 type:create cxid:0x46 zxid:0x5c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9975 T470 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 9975 T470 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 9975 T470 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:58138/sp_ffw/f/collection1/
[junit4:junit4]   2> 9975 T470 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 9976 T470 oasc.SyncStrategy.syncToMe http://127.0.0.1:58138/sp_ffw/f/collection1/ has no replicas
[junit4:junit4]   2> 9976 T470 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:58138/sp_ffw/f/collection1/
[junit4:junit4]   2> 9976 T470 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 9983 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720006 type:create cxid:0x50 zxid:0x60 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 10883 T451 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 10907 T469 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 10907 T456 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 10907 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 10960 T470 oasc.ZkController.register We are http://127.0.0.1:58138/sp_ffw/f/collection1/ and leader is http://127.0.0.1:58138/sp_ffw/f/collection1/
[junit4:junit4]   2> 10960 T470 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:58138/sp_ffw/f
[junit4:junit4]   2> 10960 T470 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 10960 T470 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 10961 T470 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 10963 T470 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 10964 T428 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 10965 T428 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 10966 T428 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 11158 T428 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 11162 T428 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:37419
[junit4:junit4]   2> 11162 T428 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 11163 T428 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 11163 T428 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360019527813
[junit4:junit4]   2> 11164 T428 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360019527813/solr.xml
[junit4:junit4]   2> 11164 T428 oasc.CoreContainer.<init> New CoreContainer 157865851
[junit4:junit4]   2> 11165 T428 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360019527813/'
[junit4:junit4]   2> 11165 T428 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360019527813/'
[junit4:junit4]   2> 11221 T428 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 11221 T428 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 11222 T428 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 11222 T428 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 11223 T428 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 11223 T428 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 11224 T428 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 11224 T428 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 11224 T428 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 11225 T428 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 11246 T428 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 11266 T428 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:45780/solr
[junit4:junit4]   2> 11267 T428 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 11267 T428 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45780 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@5bd3155b
[junit4:junit4]   2> 11269 T428 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 11269 T481 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45780. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 11270 T481 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45780, initiating session
[junit4:junit4]   2> 11270 T430 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:40216
[junit4:junit4]   2> 11270 T430 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:40216
[junit4:junit4]   2> 11273 T434 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca77aee720007 with negotiated timeout 20000 for client /140.211.11.196:40216
[junit4:junit4]   2> 11273 T481 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45780, sessionid = 0x13ca77aee720007, negotiated timeout = 20000
[junit4:junit4]   2> 11273 T482 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5bd3155b name:ZooKeeperConnection Watcher:127.0.0.1:45780 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 11273 T428 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 11275 T435 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca77aee720007
[junit4:junit4]   2> 11286 T482 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 11286 T430 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:40216 which had sessionid 0x13ca77aee720007
[junit4:junit4]   2> 11286 T428 oaz.ZooKeeper.close Session: 0x13ca77aee720007 closed
[junit4:junit4]   2> 11287 T428 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 11298 T428 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45780/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@152e63d8
[junit4:junit4]   2> 11299 T428 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 11300 T483 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45780. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 11300 T483 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45780, initiating session
[junit4:junit4]   2> 11300 T430 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:54741
[junit4:junit4]   2> 11301 T430 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:54741
[junit4:junit4]   2> 11302 T434 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca77aee720008 with negotiated timeout 20000 for client /140.211.11.196:54741
[junit4:junit4]   2> 11302 T483 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45780, sessionid = 0x13ca77aee720008, negotiated timeout = 20000
[junit4:junit4]   2> 11303 T484 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@152e63d8 name:ZooKeeperConnection Watcher:127.0.0.1:45780/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 11303 T428 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 11304 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720008 type:create cxid:0x1 zxid:0x6d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 11306 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720008 type:create cxid:0x2 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 11313 T428 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37419_sp_ffw%2Ff
[junit4:junit4]   2> 11314 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720008 type:delete cxid:0x4 zxid:0x6f txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:37419_sp_ffw%2Ff Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:37419_sp_ffw%2Ff
[junit4:junit4]   2> 11315 T428 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:37419_sp_ffw%2Ff
[junit4:junit4]   2> 11319 T469 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 11320 T456 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 11320 T469 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11320 T456 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11320 T450 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 11320 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11325 T428 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 11333 T485 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360019527813/collection1
[junit4:junit4]   2> 11333 T485 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 11334 T485 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 11334 T485 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 11336 T485 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360019527813/collection1/'
[junit4:junit4]   2> 11338 T485 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360019527813/collection1/lib/README' to classloader
[junit4:junit4]   2> 11338 T485 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360019527813/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 11393 T485 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 11467 T485 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 11568 T485 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 11576 T485 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 12240 T485 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 12252 T485 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 12256 T485 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 12283 T485 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 12289 T485 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 12294 T485 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 12296 T485 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 12298 T485 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1360019527813/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty2/
[junit4:junit4]   2> 12298 T485 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5925c9a4
[junit4:junit4]   2> 12299 T485 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 12299 T485 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty2 forceNew:false
[junit4:junit4]   2> 12300 T485 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty2
[junit4:junit4]   2> 12300 T485 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty2/index/
[junit4:junit4]   2> 12301 T485 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 12302 T485 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty2/index forceNew:false
[junit4:junit4]   2> 12313 T485 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@475d785c; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 12313 T485 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 12314 T485 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty2/index
[junit4:junit4]   2> 12315 T485 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 12317 T485 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 12317 T485 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 12317 T485 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 12318 T485 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 12319 T485 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 12319 T485 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 12319 T485 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 12320 T485 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 12320 T485 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 12321 T485 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 12322 T485 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 12322 T485 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 12323 T485 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 12323 T485 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 12323 T485 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 12324 T485 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 12324 T485 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12325 T485 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12326 T485 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12326 T485 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12327 T485 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12327 T485 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12328 T485 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12328 T485 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12329 T485 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 12330 T485 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 12342 T485 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 12347 T485 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 12348 T485 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 12351 T485 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty2
[junit4:junit4]   2> 12353 T485 oass.SolrIndexSearcher.<init> Opening Searcher@713d0cf main
[junit4:junit4]   2> 12354 T485 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 12354 T485 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 12355 T485 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 12362 T485 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 12414 T486 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@713d0cf main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 12420 T485 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 12420 T485 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 12423 T451 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12424 T451 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:58138_sp_ffw%2Ff",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:58138/sp_ffw/f"}
[junit4:junit4]   2> 12435 T451 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:37419_sp_ffw%2Ff",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:37419/sp_ffw/f"}
[junit4:junit4]   2> 12435 T451 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 12435 T451 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 12447 T456 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12447 T484 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12447 T469 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12447 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13423 T485 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 13423 T485 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:37419/sp_ffw/f collection:collection1 shard:shard2
[junit4:junit4]   2> 13424 T485 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4]   2> 13469 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720008 type:delete cxid:0x42 zxid:0x7d txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
[junit4:junit4]   2> 13470 T485 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 13471 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720008 type:create cxid:0x43 zxid:0x7e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 13481 T485 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 13481 T485 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 13481 T485 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:37419/sp_ffw/f/collection1/
[junit4:junit4]   2> 13481 T485 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 13482 T485 oasc.SyncStrategy.syncToMe http://127.0.0.1:37419/sp_ffw/f/collection1/ has no replicas
[junit4:junit4]   2> 13482 T485 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:37419/sp_ffw/f/collection1/
[junit4:junit4]   2> 13482 T485 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4]   2> 13487 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee720008 type:create cxid:0x4c zxid:0x81 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 13958 T451 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 13978 T456 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13978 T469 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13978 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13978 T484 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 14000 T485 oasc.ZkController.register We are http://127.0.0.1:37419/sp_ffw/f/collection1/ and leader is http://127.0.0.1:37419/sp_ffw/f/collection1/
[junit4:junit4]   2> 14000 T485 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:37419/sp_ffw/f
[junit4:junit4]   2> 14000 T485 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 14000 T485 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 14001 T485 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 14003 T485 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14004 T428 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 14005 T428 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 14005 T428 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 14191 T428 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 14194 T428 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:30255
[junit4:junit4]   2> 14195 T428 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 14195 T428 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 14196 T428 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360019530853
[junit4:junit4]   2> 14196 T428 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360019530853/solr.xml
[junit4:junit4]   2> 14197 T428 oasc.CoreContainer.<init> New CoreContainer 363940238
[junit4:junit4]   2> 14197 T428 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360019530853/'
[junit4:junit4]   2> 14198 T428 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360019530853/'
[junit4:junit4]   2> 14253 T428 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 14253 T428 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 14254 T428 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 14254 T428 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 14254 T428 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 14255 T428 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 14255 T428 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 14256 T428 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 14256 T428 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 14257 T428 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 14280 T428 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 14300 T428 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:45780/solr
[junit4:junit4]   2> 14301 T428 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 14301 T428 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45780 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@5c65b07d
[junit4:junit4]   2> 14302 T428 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 14303 T496 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45780. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 14304 T496 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45780, initiating session
[junit4:junit4]   2> 14304 T430 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:12017
[junit4:junit4]   2> 14305 T430 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:12017
[junit4:junit4]   2> 14307 T434 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca77aee720009 with negotiated timeout 20000 for client /140.211.11.196:12017
[junit4:junit4]   2> 14307 T496 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45780, sessionid = 0x13ca77aee720009, negotiated timeout = 20000
[junit4:junit4]   2> 14307 T497 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5c65b07d name:ZooKeeperConnection Watcher:127.0.0.1:45780 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 14307 T428 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 14309 T435 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca77aee720009
[junit4:junit4]   2> 14322 T497 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 14322 T430 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:12017 which had sessionid 0x13ca77aee720009
[junit4:junit4]   2> 14322 T428 oaz.ZooKeeper.close Session: 0x13ca77aee720009 closed
[junit4:junit4]   2> 14323 T428 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 14337 T428 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45780/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@21b9d8f4
[junit4:junit4]   2> 14338 T428 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 14338 T498 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45780. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 14338 T498 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45780, initiating session
[junit4:junit4]   2> 14339 T430 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:15905
[junit4:junit4]   2> 14339 T430 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:15905
[junit4:junit4]   2> 14340 T434 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca77aee72000a with negotiated timeout 20000 for client /140.211.11.196:15905
[junit4:junit4]   2> 14340 T498 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45780, sessionid = 0x13ca77aee72000a, negotiated timeout = 20000
[junit4:junit4]   2> 14341 T499 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@21b9d8f4 name:ZooKeeperConnection Watcher:127.0.0.1:45780/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 14341 T428 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 14342 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee72000a type:create cxid:0x1 zxid:0x8e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 14344 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee72000a type:create cxid:0x2 zxid:0x8f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 14351 T428 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:30255_sp_ffw%2Ff
[junit4:junit4]   2> 14352 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee72000a type:delete cxid:0x4 zxid:0x90 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:30255_sp_ffw%2Ff Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:30255_sp_ffw%2Ff
[junit4:junit4]   2> 14353 T428 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:30255_sp_ffw%2Ff
[junit4:junit4]   2> 14356 T484 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 14357 T456 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 14357 T469 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 14357 T456 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14357 T450 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 14358 T469 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14358 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14359 T484 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 14363 T428 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 14371 T500 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360019530853/collection1
[junit4:junit4]   2> 14371 T500 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 14372 T500 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 14372 T500 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 14374 T500 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360019530853/collection1/'
[junit4:junit4]   2> 14376 T500 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360019530853/collection1/lib/README' to classloader
[junit4:junit4]   2> 14376 T500 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360019530853/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 14431 T500 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 14506 T500 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 14607 T500 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 14615 T500 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 15272 T500 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 15285 T500 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 15289 T500 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 15315 T500 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 15321 T500 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 15326 T500 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 15328 T500 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 15330 T500 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1360019530853/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3/
[junit4:junit4]   2> 15330 T500 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5925c9a4
[junit4:junit4]   2> 15331 T500 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 15331 T500 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3 forceNew:false
[junit4:junit4]   2> 15332 T500 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3
[junit4:junit4]   2> 15332 T500 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3/index/
[junit4:junit4]   2> 15333 T500 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 15334 T500 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3/index forceNew:false
[junit4:junit4]   2> 15341 T500 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@57b3153c; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 15341 T500 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 15342 T500 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3/index
[junit4:junit4]   2> 15342 T500 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 15345 T500 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 15345 T500 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 15345 T500 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 15346 T500 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 15346 T500 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 15347 T500 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 15347 T500 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 15348 T500 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 15348 T500 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 15349 T500 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 15349 T500 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 15350 T500 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 15351 T500 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 15351 T500 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 15351 T500 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 15352 T500 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 15352 T500 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15353 T500 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15353 T500 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15354 T500 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15355 T500 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15355 T500 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15356 T500 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15356 T500 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15357 T500 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 15357 T500 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 15370 T500 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 15375 T500 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 15376 T500 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 15379 T500 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3
[junit4:junit4]   2> 15381 T500 oass.SolrIndexSearcher.<init> Opening Searcher@58c9d231 main
[junit4:junit4]   2> 15382 T500 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 15382 T500 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 15383 T500 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 15390 T500 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 15446 T501 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@58c9d231 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 15452 T500 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 15452 T500 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 15492 T451 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 15494 T451 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:37419_sp_ffw%2Ff",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:37419/sp_ffw/f"}
[junit4:junit4]   2> 15499 T451 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:30255_sp_ffw%2Ff",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:30255/sp_ffw/f"}
[junit4:junit4]   2> 15499 T451 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 15500 T451 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 15518 T456 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15518 T469 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15518 T484 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15518 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15518 T499 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16455 T500 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 16455 T500 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:30255/sp_ffw/f collection:collection1 shard:shard1
[junit4:junit4]   2> 16486 T500 oasc.ZkController.register We are http://127.0.0.1:30255/sp_ffw/f/collection1/ and leader is http://127.0.0.1:58138/sp_ffw/f/collection1/
[junit4:junit4]   2> 16486 T500 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:30255/sp_ffw/f
[junit4:junit4]   2> 16486 T500 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 16487 T500 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C14 name=collection1 org.apache.solr.core.SolrCore@3e2cc207 url=http://127.0.0.1:30255/sp_ffw/f/collection1 node=127.0.0.1:30255_sp_ffw%2Ff C14_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:30255_sp_ffw%2Ff, base_url=http://127.0.0.1:30255/sp_ffw/f}
[junit4:junit4]   2> 16494 T502 C14 P30255 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 16494 T500 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16495 T502 C14 P30255 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 16495 T502 C14 P30255 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 16495 T502 C14 P30255 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 16496 T428 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 16496 T428 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 16497 T502 C14 P30255 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 16497 T428 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 16694 T428 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 16697 T428 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:55162
[junit4:junit4]   2> 16698 T428 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 16698 T428 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 16699 T428 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360019533349
[junit4:junit4]   2> 16699 T428 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360019533349/solr.xml
[junit4:junit4]   2> 16700 T428 oasc.CoreContainer.<init> New CoreContainer 63645884
[junit4:junit4]   2> 16700 T428 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360019533349/'
[junit4:junit4]   2> 16701 T428 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360019533349/'
[junit4:junit4]   2> 16769 T428 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 16770 T428 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 16770 T428 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 16771 T428 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 16771 T428 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 16772 T428 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 16772 T428 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 16773 T428 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 16773 T428 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 16774 T428 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 16796 T428 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 16816 T428 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:45780/solr
[junit4:junit4]   2> 16817 T428 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 16817 T428 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45780 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@3d5bc62e
[junit4:junit4]   2> 16818 T428 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 16819 T512 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45780. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 16820 T512 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45780, initiating session
[junit4:junit4]   2> 16820 T430 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:47360
[junit4:junit4]   2> 16821 T430 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:47360
[junit4:junit4]   2> 16855 T434 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca77aee72000b with negotiated timeout 20000 for client /140.211.11.196:47360
[junit4:junit4]   2> 16856 T512 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45780, sessionid = 0x13ca77aee72000b, negotiated timeout = 20000
[junit4:junit4]   2> 16856 T513 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3d5bc62e name:ZooKeeperConnection Watcher:127.0.0.1:45780 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 16856 T428 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 16858 T435 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca77aee72000b
[junit4:junit4]   2> 16871 T513 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 16871 T430 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:47360 which had sessionid 0x13ca77aee72000b
[junit4:junit4]   2> 16871 T428 oaz.ZooKeeper.close Session: 0x13ca77aee72000b closed
[junit4:junit4]   2> 16872 T428 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 16883 T428 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:45780/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@3155ba4
[junit4:junit4]   2> 16884 T428 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 16884 T514 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:45780. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 16885 T514 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:45780, initiating session
[junit4:junit4]   2> 16885 T430 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:49058
[junit4:junit4]   2> 16886 T430 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:49058
[junit4:junit4]   2> 16887 T434 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca77aee72000c with negotiated timeout 20000 for client /140.211.11.196:49058
[junit4:junit4]   2> 16887 T514 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:45780, sessionid = 0x13ca77aee72000c, negotiated timeout = 20000
[junit4:junit4]   2> 16887 T515 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3155ba4 name:ZooKeeperConnection Watcher:127.0.0.1:45780/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 16888 T428 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 16889 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee72000c type:create cxid:0x1 zxid:0xa0 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 16890 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee72000c type:create cxid:0x2 zxid:0xa1 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 16898 T428 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55162_sp_ffw%2Ff
[junit4:junit4]   2> 16899 T435 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca77aee72000c type:delete cxid:0x4 zxid:0xa2 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:55162_sp_ffw%2Ff Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:55162_sp_ffw%2Ff
[junit4:junit4]   2> 16900 T428 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:55162_sp_ffw%2Ff
[junit4:junit4]   2> 16904 T484 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16905 T456 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 16905 T450 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 16905 T456 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16905 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16905 T499 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 16905 T469 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 16906 T499 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16906 T469 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16908 T484 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 16912 T428 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 16920 T516 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360019533349/collection1
[junit4:junit4]   2> 16921 T516 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 16922 T516 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 16922 T516 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 16924 T516 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360019533349/collection1/'
[junit4:junit4]   2> 16925 T516 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360019533349/collection1/lib/README' to classloader
[junit4:junit4]   2> 16926 T516 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360019533349/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 16981 T516 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 17059 T516 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 17062 T451 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17063 T451 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:30255_sp_ffw%2Ff",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:30255/sp_ffw/f"}
[junit4:junit4]   2> 17077 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17077 T499 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17077 T484 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17077 T469 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17077 T456 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17160 T516 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 17160 T515 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17168 T516 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 17833 T516 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 17848 T516 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 17852 T516 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 17889 T516 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 17895 T516 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 17902 T516 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 17904 T516 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 17906 T516 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1360019533349/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty4/
[junit4:junit4]   2> 17906 T516 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5925c9a4
[junit4:junit4]   2> 17907 T516 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 17908 T516 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty4 forceNew:false
[junit4:junit4]   2> 17909 T516 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty4
[junit4:junit4]   2> 17910 T516 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty4/index/
[junit4:junit4]   2> 17911 T516 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 17913 T516 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty4/index forceNew:false
[junit4:junit4]   2> 17931 T516 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2092ab58; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 17931 T516 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 17932 T516 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty4/index
[junit4:junit4]   2> 17933 T516 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 17935 T516 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 17935 T516 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 17936 T516 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 17936 T516 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 17937 T516 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 17937 T516 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 17937 T516 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 17938 T516 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 17939 T516 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 17939 T516 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 17940 T516 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 17940 T516 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 17941 T516 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 17941 T516 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 17941 T516 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 17942 T516 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 17943 T516 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 17943 T516 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 17944 T516 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 17944 T516 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 17945 T516 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 17945 T516 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 17946 T516 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 17947 T516 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 17947 T516 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 17948 T516 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 17969 T516 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 17976 T516 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 17978 T516 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 17982 T516 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty4
[junit4:junit4]   2> 17985 T516 oass.SolrIndexSearcher.<init> Opening Searcher@2b5bd32c main
[junit4:junit4]   2> 17986 T516 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 17986 T516 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 17987 T516 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 17995 T516 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 18048 T517 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2b5bd32c main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 18054 T516 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 18054 T516 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 18585 T451 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 18587 T451 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:55162_sp_ffw%2Ff",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:55162/sp_ffw/f"}
[junit4:junit4]   2> 18587 T451 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 18587 T451 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 18602 T456 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18602 T469 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18602 T484 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18602 T499 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18602 T515 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18602 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19057 T516 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 19057 T516 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:55162/sp_ffw/f collection:collection1 shard:shard2
[junit4:junit4]   2> 19062 T516 oasc.ZkController.register We are http://127.0.0.1:55162/sp_ffw/f/collection1/ and leader is http://127.0.0.1:37419/sp_ffw/f/collection1/
[junit4:junit4]   2> 19062 T516 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:55162/sp_ffw/f
[junit4:junit4]   2> 19062 T516 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 19062 T516 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C15 name=collection1 org.apache.solr.core.SolrCore@41d5df31 url=http://127.0.0.1:55162/sp_ffw/f/collection1 node=127.0.0.1:55162_sp_ffw%2Ff C15_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:55162_sp_ffw%2Ff, base_url=http://127.0.0.1:55162/sp_ffw/f}
[junit4:junit4]   2> 19063 T518 C15 P55162 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 19063 T518 C15 P55162 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 19064 T516 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19064 T518 C15 P55162 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 19064 T518 C15 P55162 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 19065 T428 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 19066 T518 C15 P55162 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 19066 T428 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 19066 T428 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 19085 T428 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19087 T428 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4]   2> 19088 T428 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C14_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:30255_sp_ffw%2Ff, base_url=http://127.0.0.1:30255/sp_ffw/f}
[junit4:junit4]   2> 19539 T502 C14 P30255 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:58138/sp_ffw/f/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 19548 T502 C14 P30255 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 19566 T502 C14 P30255 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:30255/sp_ffw/f START replicas=[http://127.0.0.1:58138/sp_ffw/f/collection1/] nUpdates=100
[junit4:junit4]   2> 19567 T502 C14 P30255 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 19568 T502 C14 P30255 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 19568 T502 C14 P30255 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 19568 T502 C14 P30255 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 19569 T502 C14 P30255 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 19569 T502 C14 P30255 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:58138/sp_ffw/f/collection1/. core=collection1
[junit4:junit4]   2> 19569 T502 C14 P30255 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C16 name=collection1 org.apache.solr.core.SolrCore@60a5494 url=http://127.0.0.1:58138/sp_ffw/f/collection1 node=127.0.0.1:58138_sp_ffw%2Ff C16_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:58138_sp_ffw%2Ff, base_url=http://127.0.0.1:58138/sp_ffw/f, leader=true}
[junit4:junit4]   2> 19580 T460 C16 P58138 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1 
[junit4:junit4]   2> 19599 T461 C16 P58138 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 19600 T461 C16 P58138 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty1
[junit4:junit4]   2> 19604 T461 C16 P58138 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3fb6de9f; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 19605 T461 C16 P58138 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 19635 T461 C16 P58138 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3fb6de9f; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3fb6de9f; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 19635 T461 C16 P58138 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 19637 T461 C16 P58138 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty1
[junit4:junit4]   2> 19637 T461 C16 P58138 oass.SolrIndexSearcher.<init> Opening Searcher@7765731d realtime
[junit4:junit4]   2> 19638 T461 C16 P58138 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 19638 T461 C16 P58138 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 56
[junit4:junit4]   2> 19696 T502 C14 P30255 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 19697 T502 C14 P30255 oascsi.HttpClientUtil.createClient Creating new http client, config:connTimeout=5000&socketTimeout=20000&allowCompression=false&maxConnections=10000&maxConnectionsPerHost=10000
[junit4:junit4]   2> 19723 T502 C14 P30255 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 19726 T462 C16 P58138 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 19726 T462 C16 P58138 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 19727 T502 C14 P30255 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 19728 T502 C14 P30255 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 19728 T502 C14 P30255 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 19731 T462 C16 P58138 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty1
[junit4:junit4]   2> 19732 T462 C16 P58138 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty1/index
[junit4:junit4]   2> 19732 T462 C16 P58138 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=2 
[junit4:junit4]   2> 19733 T502 C14 P30255 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 19734 T502 C14 P30255 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3/index.20130205001216565 forceNew:false
[junit4:junit4]   2> 19735 T502 C14 P30255 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3
[junit4:junit4]   2> 19735 T502 C14 P30255 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3/index.20130205001216565 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6ffa2f97; maxCacheMB=48.0 maxMergeSizeMB=4.0)) fullCopy=true
[junit4:junit4]   2> 19749 T462 C16 P58138 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=5 
[junit4:junit4]   2> 19762 T502 C14 P30255 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3/index
[junit4:junit4]   2> 19763 T502 C14 P30255 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 19764 T502 C14 P30255 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130205001216565
[junit4:junit4]   2> 19771 T502 C14 P30255 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3
[junit4:junit4]   2> 19773 T502 C14 P30255 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3
[junit4:junit4]   2> 19773 T502 C14 P30255 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3/index/ new=./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3/index.20130205001216565
[junit4:junit4]   2> 19777 T502 C14 P30255 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3/index.20130205001216565 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6ffa2f97; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 19777 T502 C14 P30255 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 19779 T502 C14 P30255 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3
[junit4:junit4]   2> 19783 T502 C14 P30255 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 19784 T502 C14 P30255 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 19784 T502 C14 P30255 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 19785 T502 C14 P30255 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3/index.20130205001216565
[junit4:junit4]   2> 19786 T502 C14 P30255 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3
[junit4:junit4]   2> 19789 T502 C14 P30255 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3/index.20130205001216565 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6ffa2f97; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 19790 T502 C14 P30255 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 19791 T502 C14 P30255 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 19791 T502 C14 P30255 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3
[junit4:junit4]   2> 19792 T502 C14 P30255 oass.SolrIndexSearcher.<init> Opening Searcher@2dd212cf main
[junit4:junit4]   2> 19793 T501 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2dd212cf main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 19794 T501 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3/index
[junit4:junit4]   2> 19795 T502 C14 P30255 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 19798 T502 C14 P30255 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3/index.20130205001216565 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6ffa2f97; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3/index.20130205001216565 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6ffa2f97; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 19799 T502 C14 P30255 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 19800 T502 C14 P30255 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3
[junit4:junit4]   2> 19800 T502 C14 P30255 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 19800 T502 C14 P30255 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3/index.20130205001216565
[junit4:junit4]   2> 19801 T502 C14 P30255 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1360019516837/jetty3/index
[junit4:junit4]   2> 19801 T502 C14 P30255 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 19801 T502 C14 P30255 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 19802 T502 C14 P30255 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 19802 T502 C14 P30255 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 19804 T502 C14 P30255 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 20092 T428 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 20111 T451 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 20112 T451 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:55162_sp_ffw%2Ff",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:55162/sp_ffw/f"}
[junit4:junit4]   2> 20118 T451 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:30255_sp_ffw%2Ff",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:30255/sp_ffw/f"}
[junit4:junit4]   2> 20130 T456 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20131 T515 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20131 T484 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20131 T499 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20131 T469 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20130 T450 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 21095 T428 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 22098 T428 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C15_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:55162_sp_ffw%2Ff, base_url=http://127.0.0.1:55162/sp_ffw/f}
[junit4:junit4]   2> 23089 T518 C15 P55162 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:37419/sp_ffw/f/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 23089 T518 C15 P55162 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:55162/sp_ffw/f START replicas=[http://127.0.0.1:37419/sp_ffw/f/collection1/] nUpdates=100
[junit4:junit4]   2> 23090 T518 C15 P55162 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 23090 T518 C15 P55162 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 23090 T518 C15 P55162 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 23091 T518 C15 P55162 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 23091 T518 C15 P55162 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 23091 T518 C15 P55162 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:37419/sp_ffw/f/collection1/. core=collection1
[junit4:junit4]   2> 23092 T518 C15 P55162 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 23101 T428 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> ASYNC  NEW_CORE C17 name=collection1 org.apache.solr.core.SolrCore@ab64f2 url=http:

[...truncated too long message...]

.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:111)
[junit4:junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:264)
[junit4:junit4]    > 	at org.apache.http.impl.conn.DefaultResponseParser.parseHead(DefaultResponseParser.java:98)
[junit4:junit4]    > 	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:252)
[junit4:junit4]    > 	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:282)
[junit4:junit4]    > 	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:247)
[junit4:junit4]    > 	at org.apache.http.impl.conn.AbstractClientConnAdapter.receiveResponseHeader(AbstractClientConnAdapter.java:216)
[junit4:junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:298)
[junit4:junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
[junit4:junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:647)
[junit4:junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:464)
[junit4:junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
[junit4:junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
[junit4:junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:732)
[junit4:junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:353)
[junit4:junit4]    > 	... 5 more
[junit4:junit4]    > Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=939, name=testExecutor-345-thread-5, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]    > Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:30255/sp_ffw/f
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([715CCD8973136CF0]:0)
[junit4:junit4]    > 	at org.apache.solr.cloud.BasicDistributedZkTest$1.run(BasicDistributedZkTest.java:658)
[junit4:junit4]    > 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
[junit4:junit4]    > 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]    > 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:30255/sp_ffw/f
[junit4:junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:412)
[junit4:junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181)
[junit4:junit4]    > 	at org.apache.solr.cloud.BasicDistributedZkTest$1.run(BasicDistributedZkTest.java:656)
[junit4:junit4]    > 	... 3 more
[junit4:junit4]    > Caused by: java.net.SocketTimeoutException: Read timed out
[junit4:junit4]    > 	at java.net.SocketInputStream.socketRead0(Native Method)
[junit4:junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:150)
[junit4:junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:121)
[junit4:junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:149)
[junit4:junit4]    > 	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:111)
[junit4:junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:264)
[junit4:junit4]    > 	at org.apache.http.impl.conn.DefaultResponseParser.parseHead(DefaultResponseParser.java:98)
[junit4:junit4]    > 	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:252)
[junit4:junit4]    > 	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:282)
[junit4:junit4]    > 	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:247)
[junit4:junit4]    > 	at org.apache.http.impl.conn.AbstractClientConnAdapter.receiveResponseHeader(AbstractClientConnAdapter.java:216)
[junit4:junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:298)
[junit4:junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
[junit4:junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:647)
[junit4:junit4]    > 	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:464)
[junit4:junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
[junit4:junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
[junit4:junit4]    > 	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:732)
[junit4:junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:353)
[junit4:junit4]    > 	... 5 more
[junit4:junit4]   2> 388062 T428 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4]   2> 388169 T431 oazs.SessionTrackerImpl.run SessionTrackerImpl exited loop!
[junit4:junit4]   2> 388258 T499 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> 388258 T499 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> 388258 T499 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 388430 T515 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> 388430 T515 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 388430 T515 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> 388430 T515 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> NOTE: test params are: codec=Lucene42: {n_f1=PostingsFormat(name=NestedPulsing), foo_b=PostingsFormat(name=Memory doPackFST= true), cat=Lucene41(blocksize=128), foo_d=Pulsing41(freqCutoff=13 minBlockSize=38 maxBlockSize=83), foo_f=PostingsFormat(name=Memory doPackFST= true), n_tl1=PostingsFormat(name=NestedPulsing), n_d1=Lucene41(blocksize=128), rnd_b=Pulsing41(freqCutoff=13 minBlockSize=38 maxBlockSize=83), intDefault=Lucene41(blocksize=128), n_td1=PostingsFormat(name=NestedPulsing), timestamp=Lucene41(blocksize=128), id=Pulsing41(freqCutoff=13 minBlockSize=38 maxBlockSize=83), range_facet_sl=PostingsFormat(name=Memory doPackFST= true), range_facet_si=PostingsFormat(name=NestedPulsing), oddField_s=PostingsFormat(name=Memory doPackFST= true), sequence_i=PostingsFormat(name=Memory doPackFST= true), name=Pulsing41(freqCutoff=13 minBlockSize=38 maxBlockSize=83), foo_i=PostingsFormat(name=NestedPulsing), regex_dup_B_s=Lucene41(blocksize=128), multiDefault=PostingsFormat(name=NestedPulsing), n_tf1=Lucene41(blocksize=128), n_dt1=PostingsFormat(name=NestedPulsing), genre_s=PostingsFormat(name=Memory doPackFST= true), author_t=PostingsFormat(name=NestedPulsing), n_ti1=Pulsing41(freqCutoff=13 minBlockSize=38 maxBlockSize=83), range_facet_l=Lucene41(blocksize=128), text=PostingsFormat(name=Memory doPackFST= true), _version_=Lucene41(blocksize=128), val_i=Pulsing41(freqCutoff=13 minBlockSize=38 maxBlockSize=83), SubjectTerms_mfacet=Lucene41(blocksize=128), series_t=PostingsFormat(name=NestedPulsing), a_t=Lucene41(blocksize=128), n_tdt1=Lucene41(blocksize=128), regex_dup_A_s=PostingsFormat(name=Memory doPackFST= true), price=PostingsFormat(name=Memory doPackFST= true), other_tl1=Lucene41(blocksize=128), n_l1=Lucene41(blocksize=128), a_si=PostingsFormat(name=NestedPulsing), inStock=PostingsFormat(name=Memory doPackFST= true)}, sim=DefaultSimilarity, locale=et, timezone=ECT
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=2,free=327414984,total=475004928
[junit4:junit4]   2> NOTE: All tests run in this JVM: [RequestHandlersTest, TestFiltering, TestRandomFaceting, TestDefaultSimilarityFactory, BinaryUpdateRequestHandlerTest, DistributedSpellCheckComponentTest, TestGroupingSearch, JsonLoaderTest, BasicDistributedZkTest]
[junit4:junit4] Completed on J0 in 388.65s, 1 test, 1 error <<< FAILURES!

[...truncated 582 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:346: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/common-build.xml:428: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:1176: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:841: There were test failures: 249 suites, 1038 tests, 1 error, 23 ignored (5 assumptions)

Total time: 39 minutes 26 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-trunk-java7 - Build # 3707 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-java7/3707/

1 tests failed.
FAILED:  org.apache.solr.cloud.BasicDistributedZkTest.testDistribSearch

Error Message:
parsing error

Stack Trace:
org.apache.solr.common.SolrException: parsing error
	at __randomizedtesting.SeedInfo.seed([62599A281191218A:E3BF143066CE41B6]:0)
	at org.apache.solr.client.solrj.impl.BinaryResponseParser.processResponse(BinaryResponseParser.java:43)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:387)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181)
	at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
	at org.apache.solr.client.solrj.SolrServer.commit(SolrServer.java:168)
	at org.apache.solr.client.solrj.SolrServer.commit(SolrServer.java:146)
	at org.apache.solr.cloud.BasicDistributedZkTest.testCoreUnloadAndLeaders(BasicDistributedZkTest.java:611)
	at org.apache.solr.cloud.BasicDistributedZkTest.doTest(BasicDistributedZkTest.java:337)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:794)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:722)
Caused by: java.net.SocketException: Socket closed
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:150)
	at java.net.SocketInputStream.read(SocketInputStream.java:121)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:149)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:111)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.read(AbstractSessionInputBuffer.java:195)
	at org.apache.http.impl.io.ContentLengthInputStream.read(ContentLengthInputStream.java:176)
	at org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:138)
	at org.apache.solr.common.util.FastInputStream.readWrappedStream(FastInputStream.java:80)
	at org.apache.solr.common.util.FastInputStream.refill(FastInputStream.java:89)
	at org.apache.solr.common.util.FastInputStream.readByte(FastInputStream.java:192)
	at org.apache.solr.common.util.JavaBinCodec.unmarshal(JavaBinCodec.java:107)
	at org.apache.solr.client.solrj.impl.BinaryResponseParser.processResponse(BinaryResponseParser.java:41)
	... 48 more




Build Log:
[...truncated 8746 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
[junit4:junit4]   2> 0 T79 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /gd_w/gh
[junit4:junit4]   2> 8 T79 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZkTest-1359986689674
[junit4:junit4]   2> 10 T79 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 10 T80 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 18 T80 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 19 T80 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4]   2> 19 T80 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4]   2> 20 T80 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 21 T80 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4]   2> 111 T79 oasc.ZkTestServer.run start zk server on port:29524
[junit4:junit4]   2> 117 T79 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:29524 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@55cc8bbe
[junit4:junit4]   2> 118 T79 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 118 T85 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:29524. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 119 T85 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:29524, initiating session
[junit4:junit4]   2> 120 T81 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:34695
[junit4:junit4]   2> 120 T81 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:34695
[junit4:junit4]   2> 121 T83 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 125 T83 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca58606970000 with negotiated timeout 10000 for client /140.211.11.196:34695
[junit4:junit4]   2> 125 T85 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:29524, sessionid = 0x13ca58606970000, negotiated timeout = 10000
[junit4:junit4]   2> 126 T86 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@55cc8bbe name:ZooKeeperConnection Watcher:127.0.0.1:29524 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 126 T79 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 127 T79 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 136 T84 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca58606970000
[junit4:junit4]   2> 137 T86 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 137 T79 oaz.ZooKeeper.close Session: 0x13ca58606970000 closed
[junit4:junit4]   2> 138 T81 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:34695 which had sessionid 0x13ca58606970000
[junit4:junit4]   2> 138 T79 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:29524/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@1de43fc7
[junit4:junit4]   2> 139 T79 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 140 T87 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:29524. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 141 T87 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:29524, initiating session
[junit4:junit4]   2> 141 T81 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:14236
[junit4:junit4]   2> 141 T81 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:14236
[junit4:junit4]   2> 143 T83 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca58606970001 with negotiated timeout 10000 for client /140.211.11.196:14236
[junit4:junit4]   2> 143 T87 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:29524, sessionid = 0x13ca58606970001, negotiated timeout = 10000
[junit4:junit4]   2> 144 T88 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1de43fc7 name:ZooKeeperConnection Watcher:127.0.0.1:29524/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 144 T79 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 145 T79 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 149 T79 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 158 T79 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 161 T79 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 165 T79 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 166 T79 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 179 T79 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 181 T79 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 289 T79 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 290 T79 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 294 T79 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 295 T79 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 306 T79 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 308 T79 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 312 T79 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 314 T79 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 318 T79 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 320 T79 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 325 T79 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 325 T79 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 329 T79 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 330 T79 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 334 T84 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca58606970001
[junit4:junit4]   2> 335 T88 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 336 T81 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:14236 which had sessionid 0x13ca58606970001
[junit4:junit4]   2> 335 T79 oaz.ZooKeeper.close Session: 0x13ca58606970001 closed
[junit4:junit4]   2> 1082 T79 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 1339 T79 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:44201
[junit4:junit4]   2> 1375 T79 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 1375 T79 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1376 T79 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359986690002
[junit4:junit4]   2> 1376 T79 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359986690002/solr.xml
[junit4:junit4]   2> 1377 T79 oasc.CoreContainer.<init> New CoreContainer 496953775
[junit4:junit4]   2> 1377 T79 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359986690002/'
[junit4:junit4]   2> 1378 T79 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359986690002/'
[junit4:junit4]   2> 1417 T79 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 1418 T79 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 1418 T79 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 1419 T79 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 1419 T79 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 1419 T79 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 1420 T79 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 1420 T79 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 1421 T79 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 1421 T79 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 1439 T79 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 1455 T79 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:29524/solr
[junit4:junit4]   2> 1456 T79 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 1456 T79 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:29524 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@42db0359
[junit4:junit4]   2> 1457 T79 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1459 T99 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:29524. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 1460 T99 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:29524, initiating session
[junit4:junit4]   2> 1460 T81 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:39110
[junit4:junit4]   2> 1461 T81 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:39110
[junit4:junit4]   2> 1467 T83 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca58606970002 with negotiated timeout 20000 for client /140.211.11.196:39110
[junit4:junit4]   2> 1467 T99 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:29524, sessionid = 0x13ca58606970002, negotiated timeout = 20000
[junit4:junit4]   2> 1468 T100 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@42db0359 name:ZooKeeperConnection Watcher:127.0.0.1:29524 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1468 T79 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1470 T84 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca58606970002
[junit4:junit4]   2> 1476 T100 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 1476 T81 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:39110 which had sessionid 0x13ca58606970002
[junit4:junit4]   2> 1476 T79 oaz.ZooKeeper.close Session: 0x13ca58606970002 closed
[junit4:junit4]   2> 1477 T79 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 1487 T79 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:29524/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@2fad4ba4
[junit4:junit4]   2> 1487 T79 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1488 T101 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:29524. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 1489 T101 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:29524, initiating session
[junit4:junit4]   2> 1489 T81 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:27981
[junit4:junit4]   2> 1489 T81 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:27981
[junit4:junit4]   2> 1491 T83 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca58606970003 with negotiated timeout 20000 for client /140.211.11.196:27981
[junit4:junit4]   2> 1491 T101 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:29524, sessionid = 0x13ca58606970003, negotiated timeout = 20000
[junit4:junit4]   2> 1492 T102 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2fad4ba4 name:ZooKeeperConnection Watcher:127.0.0.1:29524/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1492 T79 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1494 T84 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca58606970003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 1502 T79 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 1505 T79 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44201_gd_w%2Fgh
[junit4:junit4]   2> 1506 T84 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca58606970003 type:delete cxid:0x7 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:44201_gd_w%2Fgh Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:44201_gd_w%2Fgh
[junit4:junit4]   2> 1507 T79 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:44201_gd_w%2Fgh
[junit4:junit4]   2> 1512 T79 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 1527 T84 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca58606970003 type:delete cxid:0x15 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 1528 T79 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 1531 T79 oasc.Overseer.start Overseer (id=89128087695327235-127.0.0.1:44201_gd_w%2Fgh-n_0000000000) starting
[junit4:junit4]   2> 1532 T84 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca58606970003 type:create cxid:0x1a zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 1541 T84 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca58606970003 type:create cxid:0x1b zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 1542 T84 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca58606970003 type:create cxid:0x1c zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 1543 T84 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca58606970003 type:create cxid:0x1d zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 1552 T104 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 1553 T79 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 1556 T79 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 1558 T103 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 1562 T105 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359986690002/collection1
[junit4:junit4]   2> 1563 T105 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 1563 T105 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 1564 T105 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 1566 T105 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359986690002/collection1/'
[junit4:junit4]   2> 1567 T105 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359986690002/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 1567 T105 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359986690002/collection1/lib/README' to classloader
[junit4:junit4]   2> 1616 T105 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1679 T105 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1780 T105 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 1801 T105 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 2499 T105 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 2517 T105 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 2522 T105 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 2543 T105 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2548 T105 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2552 T105 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2555 T105 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2558 T105 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359986690002/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/control/data/
[junit4:junit4]   2> 2559 T105 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@36325789
[junit4:junit4]   2> 2560 T105 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 2584 T105 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/control/data forceNew:false
[junit4:junit4]   2> 2585 T105 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/control/data
[junit4:junit4]   2> 2585 T105 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/control/data/index/
[junit4:junit4]   2> 2586 T105 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 2587 T105 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/control/data/index forceNew:false
[junit4:junit4]   2> 2597 T105 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@796b0a2d; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 2597 T105 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 2598 T105 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/control/data/index
[junit4:junit4]   2> 2599 T105 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 2676 T105 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 2676 T105 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 2677 T105 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 2683 T105 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 2722 T105 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 2722 T105 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 2725 T105 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 2730 T105 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 2745 T105 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 2788 T105 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 2790 T105 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 2794 T105 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 2795 T105 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 2795 T105 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 2795 T105 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 2796 T105 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 2797 T105 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2798 T105 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2799 T105 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2800 T105 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2801 T105 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2802 T105 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2803 T105 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2804 T105 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2819 T105 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 2834 T105 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 2843 T105 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 2846 T105 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 2848 T105 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 2850 T105 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/control/data
[junit4:junit4]   2> 2851 T105 oass.SolrIndexSearcher.<init> Opening Searcher@5da4c884 main
[junit4:junit4]   2> 2874 T105 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 2874 T105 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 2875 T105 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 2920 T105 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 3017 T106 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5da4c884 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 3030 T105 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 3030 T105 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 3031 T84 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca58606970003 type:create cxid:0x50 zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 3069 T103 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3070 T103 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:44201_gd_w%2Fgh",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:44201/gd_w/gh"}
[junit4:junit4]   2> 3071 T103 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4]   2> 3094 T103 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 3095 T84 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca58606970003 type:create cxid:0x58 zxid:0x2c txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 3120 T102 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 4036 T105 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 4036 T105 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:44201/gd_w/gh collection:control_collection shard:shard1
[junit4:junit4]   2> 4051 T105 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 4070 T105 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4]   2> 4113 T84 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca58606970003 type:delete cxid:0x72 zxid:0x36 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4]   2> 4114 T105 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 4115 T84 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca58606970003 type:create cxid:0x73 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4125 T105 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 4125 T105 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 4126 T105 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:44201/gd_w/gh/collection1/
[junit4:junit4]   2> 4126 T105 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 4127 T105 oasc.SyncStrategy.syncToMe http://127.0.0.1:44201/gd_w/gh/collection1/ has no replicas
[junit4:junit4]   2> 4127 T105 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:44201/gd_w/gh/collection1/
[junit4:junit4]   2> 4127 T105 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4]   2> 4133 T84 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca58606970003 type:create cxid:0x7d zxid:0x3b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4625 T103 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 4646 T102 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 4708 T105 oasc.ZkController.register We are http://127.0.0.1:44201/gd_w/gh/collection1/ and leader is http://127.0.0.1:44201/gd_w/gh/collection1/
[junit4:junit4]   2> 4713 T105 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:44201/gd_w/gh
[junit4:junit4]   2> 4713 T105 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 4714 T105 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 4714 T105 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 4718 T105 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 4720 T79 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 4721 T79 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 4835 T79 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 4862 T79 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 4881 T79 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:29524/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@5286fcc4
[junit4:junit4]   2> 4882 T79 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4883 T107 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:29524. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4884 T107 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:29524, initiating session
[junit4:junit4]   2> 4884 T81 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:26484
[junit4:junit4]   2> 4884 T81 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:26484
[junit4:junit4]   2> 4886 T83 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca58606970004 with negotiated timeout 10000 for client /140.211.11.196:26484
[junit4:junit4]   2> 4886 T107 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:29524, sessionid = 0x13ca58606970004, negotiated timeout = 10000
[junit4:junit4]   2> 4887 T108 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5286fcc4 name:ZooKeeperConnection Watcher:127.0.0.1:29524/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4887 T79 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4889 T79 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 4894 T79 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 5081 T79 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 5086 T79 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:36837
[junit4:junit4]   2> 5087 T79 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 5088 T79 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 5089 T79 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359986694559
[junit4:junit4]   2> 5090 T79 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359986694559/solr.xml
[junit4:junit4]   2> 5091 T79 oasc.CoreContainer.<init> New CoreContainer 885644617
[junit4:junit4]   2> 5092 T79 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359986694559/'
[junit4:junit4]   2> 5093 T79 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359986694559/'
[junit4:junit4]   2> 5177 T79 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 5177 T79 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 5178 T79 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 5179 T79 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 5179 T79 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 5180 T79 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 5180 T79 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 5181 T79 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 5182 T79 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 5182 T79 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 5207 T79 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 5229 T79 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:29524/solr
[junit4:junit4]   2> 5230 T79 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 5230 T79 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:29524 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@16db431a
[junit4:junit4]   2> 5232 T79 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 5232 T118 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:29524. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 5233 T118 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:29524, initiating session
[junit4:junit4]   2> 5234 T81 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:16881
[junit4:junit4]   2> 5234 T81 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:16881
[junit4:junit4]   2> 5236 T83 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca58606970005 with negotiated timeout 20000 for client /140.211.11.196:16881
[junit4:junit4]   2> 5237 T118 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:29524, sessionid = 0x13ca58606970005, negotiated timeout = 20000
[junit4:junit4]   2> 5237 T119 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@16db431a name:ZooKeeperConnection Watcher:127.0.0.1:29524 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 5238 T79 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 5240 T84 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca58606970005
[junit4:junit4]   2> 5241 T119 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 5241 T81 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:16881 which had sessionid 0x13ca58606970005
[junit4:junit4]   2> 5241 T79 oaz.ZooKeeper.close Session: 0x13ca58606970005 closed
[junit4:junit4]   2> 5242 T79 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 5256 T79 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:29524/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@8279c5e
[junit4:junit4]   2> 5257 T79 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 5258 T120 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:29524. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 5259 T120 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:29524, initiating session
[junit4:junit4]   2> 5259 T81 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:29257
[junit4:junit4]   2> 5259 T81 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:29257
[junit4:junit4]   2> 5261 T83 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca58606970006 with negotiated timeout 20000 for client /140.211.11.196:29257
[junit4:junit4]   2> 5261 T120 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:29524, sessionid = 0x13ca58606970006, negotiated timeout = 20000
[junit4:junit4]   2> 5262 T121 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@8279c5e name:ZooKeeperConnection Watcher:127.0.0.1:29524/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 5262 T79 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 5264 T84 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca58606970006 type:create cxid:0x1 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5266 T84 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca58606970006 type:create cxid:0x2 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5268 T79 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36837_gd_w%2Fgh
[junit4:junit4]   2> 5269 T84 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca58606970006 type:delete cxid:0x4 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:36837_gd_w%2Fgh Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:36837_gd_w%2Fgh
[junit4:junit4]   2> 5270 T79 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:36837_gd_w%2Fgh
[junit4:junit4]   2> 5273 T102 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 5274 T108 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 5282 T102 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 5288 T79 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 5297 T122 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359986694559/collection1
[junit4:junit4]   2> 5298 T122 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 5299 T122 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 5299 T122 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 5301 T122 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359986694559/collection1/'
[junit4:junit4]   2> 5303 T122 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359986694559/collection1/lib/README' to classloader
[junit4:junit4]   2> 5304 T122 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359986694559/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 5383 T122 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 5464 T122 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 5566 T122 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 5595 T122 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 6159 T103 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6162 T103 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:44201_gd_w%2Fgh",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:44201/gd_w/gh"}
[junit4:junit4]   2> 6224 T108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6224 T121 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6224 T102 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6316 T122 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 6334 T122 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 6339 T122 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 6370 T122 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 6377 T122 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 6383 T122 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 6387 T122 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 6390 T122 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359986694559/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty1/
[junit4:junit4]   2> 6391 T122 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@36325789
[junit4:junit4]   2> 6393 T122 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 6394 T122 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty1 forceNew:false
[junit4:junit4]   2> 6395 T122 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty1
[junit4:junit4]   2> 6395 T122 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty1/index/
[junit4:junit4]   2> 6396 T122 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 6397 T122 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty1/index forceNew:false
[junit4:junit4]   2> 6407 T122 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@396cd7c2; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 6408 T122 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 6409 T122 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty1/index
[junit4:junit4]   2> 6409 T122 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 6413 T122 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 6413 T122 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 6414 T122 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 6415 T122 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 6416 T122 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 6416 T122 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 6417 T122 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 6420 T122 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 6422 T122 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 6424 T122 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 6426 T122 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 6427 T122 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 6429 T122 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 6429 T122 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 6430 T122 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 6432 T122 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 6434 T122 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6436 T122 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6438 T122 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6440 T122 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6442 T122 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6444 T122 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6446 T122 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6448 T122 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6450 T122 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 6452 T122 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 6465 T122 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 6470 T122 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 6473 T122 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 6476 T122 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty1
[junit4:junit4]   2> 6478 T122 oass.SolrIndexSearcher.<init> Opening Searcher@7aea68c0 main
[junit4:junit4]   2> 6479 T122 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 6480 T122 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 6481 T122 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 6495 T122 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 6563 T123 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7aea68c0 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 6584 T122 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 6584 T122 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 7778 T103 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 7780 T103 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:36837_gd_w%2Fgh",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:36837/gd_w/gh"}
[junit4:junit4]   2> 7780 T103 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4]   2> 7780 T103 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 7806 T108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7806 T121 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7806 T102 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8589 T122 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 8589 T122 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:36837/gd_w/gh collection:collection1 shard:shard1
[junit4:junit4]   2> 8590 T122 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 8635 T84 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca58606970006 type:delete cxid:0x45 zxid:0x5b txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 8636 T122 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 8636 T84 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca58606970006 type:create cxid:0x46 zxid:0x5c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8646 T122 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 8647 T122 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 8647 T122 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:36837/gd_w/gh/collection1/
[junit4:junit4]   2> 8647 T122 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 8648 T122 oasc.SyncStrategy.syncToMe http://127.0.0.1:36837/gd_w/gh/collection1/ has no replicas
[junit4:junit4]   2> 8648 T122 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:36837/gd_w/gh/collection1/
[junit4:junit4]   2> 8649 T122 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 8654 T84 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca58606970006 type:create cxid:0x50 zxid:0x60 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9312 T103 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9339 T108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 9339 T121 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 9339 T102 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 9378 T122 oasc.ZkController.register We are http://127.0.0.1:36837/gd_w/gh/collection1/ and leader is http://127.0.0.1:36837/gd_w/gh/collection1/
[junit4:junit4]   2> 9378 T122 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:36837/gd_w/gh
[junit4:junit4]   2> 9378 T122 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 9379 T122 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 9379 T122 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 9382 T122 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9384 T79 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 9385 T79 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 9385 T79 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 9574 T79 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 9578 T79 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:23227
[junit4:junit4]   2> 9579 T79 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 9580 T79 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 9580 T79 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359986699069
[junit4:junit4]   2> 9581 T79 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359986699069/solr.xml
[junit4:junit4]   2> 9581 T79 oasc.CoreContainer.<init> New CoreContainer 303487423
[junit4:junit4]   2> 9582 T79 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359986699069/'
[junit4:junit4]   2> 9583 T79 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359986699069/'
[junit4:junit4]   2> 9641 T79 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 9641 T79 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 9642 T79 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 9643 T79 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 9643 T79 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 9644 T79 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 9644 T79 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 9645 T79 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 9645 T79 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 9646 T79 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 9670 T79 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 9692 T79 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:29524/solr
[junit4:junit4]   2> 9693 T79 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 9694 T79 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:29524 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@5c055c1e
[junit4:junit4]   2> 9695 T79 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 9695 T133 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:29524. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 9696 T133 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:29524, initiating session
[junit4:junit4]   2> 9697 T81 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:19140
[junit4:junit4]   2> 9697 T81 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:19140
[junit4:junit4]   2> 9699 T83 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca58606970007 with negotiated timeout 20000 for client /140.211.11.196:19140
[junit4:junit4]   2> 9699 T133 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:29524, sessionid = 0x13ca58606970007, negotiated timeout = 20000
[junit4:junit4]   2> 9700 T134 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5c055c1e name:ZooKeeperConnection Watcher:127.0.0.1:29524 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 9700 T79 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 9702 T84 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca58606970007
[junit4:junit4]   2> 9703 T134 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 9703 T81 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:19140 which had sessionid 0x13ca58606970007
[junit4:junit4]   2> 9703 T79 oaz.ZooKeeper.close Session: 0x13ca58606970007 closed
[junit4:junit4]   2> 9704 T79 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 9716 T79 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:29524/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@24f076a0
[junit4:junit4]   2> 9717 T79 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 9718 T135 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:29524. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 9718 T135 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:29524, initiating session
[junit4:junit4]   2> 9719 T81 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:30928
[junit4:junit4]   2> 9719 T81 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:30928
[junit4:junit4]   2> 9721 T83 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca58606970008 with negotiated timeout 20000 for client /140.211.11.196:30928
[junit4:junit4]   2> 9721 T135 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:29524, sessionid = 0x13ca58606970008, negotiated timeout = 20000
[junit4:junit4]   2> 9721 T136 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@24f076a0 name:ZooKeeperConnection Watcher:127.0.0.1:29524/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 9722 T79 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 9724 T84 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca58606970008 type:create cxid:0x1 zxid:0x6d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9725 T84 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca58606970008 type:create cxid:0x2 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9727 T79 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:23227_gd_w%2Fgh
[junit4:junit4]   2> 9728 T84 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca58606970008 type:delete cxid:0x4 zxid:0x6f txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:23227_gd_w%2Fgh Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:23227_gd_w%2Fgh
[junit4:junit4]   2> 9729 T79 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:23227_gd_w%2Fgh
[junit4:junit4]   2> 9738 T121 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 9738 T102 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 9738 T108 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 9739 T108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9740 T102 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 9741 T121 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 9743 T79 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 9752 T137 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359986699069/collection1
[junit4:junit4]   2> 9752 T137 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 9753 T137 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 9753 T137 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 9755 T137 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359986699069/collection1/'
[junit4:junit4]   2> 9757 T137 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359986699069/collection1/lib/README' to classloader
[junit4:junit4]   2> 9758 T137 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359986699069/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 9822 T137 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 9903 T137 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 10004 T137 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 10025 T137 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 10664 T137 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 10676 T137 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 10679 T137 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 10701 T137 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 10706 T137 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 10710 T137 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 10712 T137 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 10714 T137 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359986699069/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty2/
[junit4:junit4]   2> 10715 T137 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@36325789
[junit4:junit4]   2> 10715 T137 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 10716 T137 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty2 forceNew:false
[junit4:junit4]   2> 10717 T137 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty2
[junit4:junit4]   2> 10717 T137 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty2/index/
[junit4:junit4]   2> 10717 T137 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 10718 T137 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty2/index forceNew:false
[junit4:junit4]   2> 10723 T137 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@547b471e; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 10724 T137 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 10725 T137 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty2/index
[junit4:junit4]   2> 10725 T137 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 10728 T137 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 10728 T137 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 10728 T137 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 10729 T137 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 10729 T137 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 10730 T137 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 10730 T137 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 10731 T137 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 10732 T137 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 10734 T137 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 10736 T137 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 10737 T137 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 10739 T137 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 10739 T137 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 10740 T137 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 10742 T137 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 10744 T137 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10745 T137 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10747 T137 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10749 T137 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10751 T137 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10753 T137 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10755 T137 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10757 T137 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10759 T137 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 10761 T137 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 10774 T137 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 10779 T137 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 10781 T137 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 10784 T137 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty2
[junit4:junit4]   2> 10786 T137 oass.SolrIndexSearcher.<init> Opening Searcher@69b92fd3 main
[junit4:junit4]   2> 10787 T137 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 10788 T137 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 10789 T137 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 10795 T137 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 10847 T103 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 10849 T103 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:36837_gd_w%2Fgh",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:36837/gd_w/gh"}
[junit4:junit4]   2> 10853 T138 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@69b92fd3 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 10855 T136 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10855 T102 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10855 T108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10855 T121 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10862 T137 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 10862 T137 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 12369 T103 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12371 T103 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:23227_gd_w%2Fgh",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:23227/gd_w/gh"}
[junit4:junit4]   2> 12371 T103 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 12371 T103 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 12412 T102 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12412 T121 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12412 T136 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12412 T108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12868 T137 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 12868 T137 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:23227/gd_w/gh collection:collection1 shard:shard2
[junit4:junit4]   2> 12870 T137 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4]   2> 12890 T84 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca58606970008 type:delete cxid:0x44 zxid:0x7e txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
[junit4:junit4]   2> 12891 T137 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 12892 T84 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca58606970008 type:create cxid:0x45 zxid:0x7f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12903 T137 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 12903 T137 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 12903 T137 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:23227/gd_w/gh/collection1/
[junit4:junit4]   2> 12904 T137 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 12904 T137 oasc.SyncStrategy.syncToMe http://127.0.0.1:23227/gd_w/gh/collection1/ has no replicas
[junit4:junit4]   2> 12904 T137 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:23227/gd_w/gh/collection1/
[junit4:junit4]   2> 12905 T137 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4]   2> 12914 T84 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca58606970008 type:create cxid:0x4e zxid:0x82 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 13918 T103 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 13937 T102 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13937 T121 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13937 T108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13937 T136 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13997 T137 oasc.ZkController.register We are http://127.0.0.1:23227/gd_w/gh/collection1/ and leader is http://127.0.0.1:23227/gd_w/gh/collection1/
[junit4:junit4]   2> 13997 T137 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:23227/gd_w/gh
[junit4:junit4]   2> 13997 T137 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 13998 T137 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 13998 T137 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 14000 T137 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14002 T79 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 14002 T79 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 14003 T79 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 14216 T79 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 14220 T79 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:26927
[junit4:junit4]   2> 14221 T79 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 14221 T79 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 14222 T79 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359986703684
[junit4:junit4]   2> 14222 T79 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359986703684/solr.xml
[junit4:junit4]   2> 14223 T79 oasc.CoreContainer.<init> New CoreContainer 956099114
[junit4:junit4]   2> 14224 T79 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359986703684/'
[junit4:junit4]   2> 14224 T79 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359986703684/'
[junit4:junit4]   2> 14299 T79 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 14300 T79 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 14300 T79 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 14301 T79 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 14301 T79 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 14302 T79 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 14303 T79 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 14303 T79 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 14304 T79 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 14304 T79 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 14330 T79 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 14355 T79 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:29524/solr
[junit4:junit4]   2> 14355 T79 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 14356 T79 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:29524 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@20885af
[junit4:junit4]   2> 14357 T79 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 14358 T148 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:29524. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 14359 T148 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:29524, initiating session
[junit4:junit4]   2> 14359 T81 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:30584
[junit4:junit4]   2> 14360 T81 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:30584
[junit4:junit4]   2> 14362 T83 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca58606970009 with negotiated timeout 20000 for client /140.211.11.196:30584
[junit4:junit4]   2> 14362 T148 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:29524, sessionid = 0x13ca58606970009, negotiated timeout = 20000
[junit4:junit4]   2> 14363 T149 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@20885af name:ZooKeeperConnection Watcher:127.0.0.1:29524 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 14363 T79 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 14365 T84 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca58606970009
[junit4:junit4]   2> 14367 T149 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 14367 T81 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:30584 which had sessionid 0x13ca58606970009
[junit4:junit4]   2> 14367 T79 oaz.ZooKeeper.close Session: 0x13ca58606970009 closed
[junit4:junit4]   2> 14368 T79 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 14380 T79 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:29524/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@404c6509
[junit4:junit4]   2> 14381 T79 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 14381 T150 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:29524. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 14382 T150 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:29524, initiating session
[junit4:junit4]   2> 14382 T81 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:19456
[junit4:junit4]   2> 14383 T81 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:19456
[junit4:junit4]   2> 14385 T83 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca5860697000a with negotiated timeout 20000 for client /140.211.11.196:19456
[junit4:junit4]   2> 14385 T150 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:29524, sessionid = 0x13ca5860697000a, negotiated timeout = 20000
[junit4:junit4]   2> 14385 T151 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@404c6509 name:ZooKeeperConnection Watcher:127.0.0.1:29524/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 14386 T79 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 14387 T84 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca5860697000a type:create cxid:0x1 zxid:0x8f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 14388 T84 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca5860697000a type:create cxid:0x2 zxid:0x90 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 14390 T79 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:26927_gd_w%2Fgh
[junit4:junit4]   2> 14391 T84 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca5860697000a type:delete cxid:0x4 zxid:0x91 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:26927_gd_w%2Fgh Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:26927_gd_w%2Fgh
[junit4:junit4]   2> 14397 T79 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:26927_gd_w%2Fgh
[junit4:junit4]   2> 14400 T102 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 14400 T121 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 14400 T136 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 14400 T108 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 14401 T136 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14401 T108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14402 T102 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 14404 T121 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 14405 T79 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 14424 T152 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359986703684/collection1
[junit4:junit4]   2> 14425 T152 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 14426 T152 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 14426 T152 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 14428 T152 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359986703684/collection1/'
[junit4:junit4]   2> 14430 T152 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359986703684/collection1/lib/README' to classloader
[junit4:junit4]   2> 14431 T152 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359986703684/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 14495 T152 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 14578 T152 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 14679 T152 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 14697 T152 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 15427 T152 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 15444 T152 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 15448 T152 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 15451 T103 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 15454 T103 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:23227_gd_w%2Fgh",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:23227/gd_w/gh"}
[junit4:junit4]   2> 15462 T102 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15462 T151 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15462 T108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15462 T121 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15462 T136 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15488 T152 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 15495 T152 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 15502 T152 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 15505 T152 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 15508 T152 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359986703684/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty3/
[junit4:junit4]   2> 15509 T152 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@36325789
[junit4:junit4]   2> 15510 T152 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 15511 T152 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty3 forceNew:false
[junit4:junit4]   2> 15512 T152 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty3
[junit4:junit4]   2> 15512 T152 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty3/index/
[junit4:junit4]   2> 15513 T152 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 15514 T152 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty3/index forceNew:false
[junit4:junit4]   2> 15521 T152 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@60912875; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 15522 T152 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 15523 T152 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty3/index
[junit4:junit4]   2> 15523 T152 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 15527 T152 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 15528 T152 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 15528 T152 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 15529 T152 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 15530 T152 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 15530 T152 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 15531 T152 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 15533 T152 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 15534 T152 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 15535 T152 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 15537 T152 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 15537 T152 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 15539 T152 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 15539 T152 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 15540 T152 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 15541 T152 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 15542 T152 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15544 T152 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15545 T152 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15546 T152 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15547 T152 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15549 T152 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15550 T152 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15551 T152 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15553 T152 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 15554 T152 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 15569 T152 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 15574 T152 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 15576 T152 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 15579 T152 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty3
[junit4:junit4]   2> 15581 T152 oass.SolrIndexSearcher.<init> Opening Searcher@1ce2dd88 main
[junit4:junit4]   2> 15583 T152 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 15583 T152 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 15584 T152 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 15593 T152 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 15657 T153 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1ce2dd88 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 15673 T152 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 15673 T152 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 16974 T103 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16976 T103 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:26927_gd_w%2Fgh",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:26927/gd_w/gh"}
[junit4:junit4]   2> 16976 T103 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 16977 T103 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 16995 T102 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16995 T136 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16995 T108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16995 T121 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16995 T151 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 17678 T152 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 17678 T152 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:26927/gd_w/gh collection:collection1 shard:shard1
[junit4:junit4]   2> 17682 T152 oasc.ZkController.register We are http://127.0.0.1:26927/gd_w/gh/collection1/ and leader is http://127.0.0.1:36837/gd_w/gh/collection1/
[junit4:junit4]   2> 17683 T152 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:26927/gd_w/gh
[junit4:junit4]   2> 17683 T152 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 17684 T152 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C2 name=collection1 org.apache.solr.core.SolrCore@65c4d40c url=http://127.0.0.1:26927/gd_w/gh/collection1 node=127.0.0.1:26927_gd_w%2Fgh C2_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:26927_gd_w%2Fgh, base_url=http://127.0.0.1:26927/gd_w/gh}
[junit4:junit4]   2> 17690 T154 C2 P26927 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 17691 T152 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17691 T154 C2 P26927 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 17692 T154 C2 P26927 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 17692 T154 C2 P26927 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 17693 T79 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 17693 T79 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 17694 T79 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 17697 T154 C2 P26927 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 17972 T79 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 17976 T79 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:48723
[junit4:junit4]   2> 17977 T79 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 17977 T79 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 17978 T79 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359986707385
[junit4:junit4]   2> 17978 T79 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359986707385/solr.xml
[junit4:junit4]   2> 17979 T79 oasc.CoreContainer.<init> New CoreContainer 374310541
[junit4:junit4]   2> 17980 T79 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359986707385/'
[junit4:junit4]   2> 17981 T79 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359986707385/'
[junit4:junit4]   2> 18069 T79 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 18070 T79 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 18070 T79 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 18071 T79 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 18071 T79 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 18072 T79 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 18072 T79 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 18073 T79 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 18074 T79 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 18074 T79 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 18103 T79 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 18130 T79 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:29524/solr
[junit4:junit4]   2> 18130 T79 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 18131 T79 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:29524 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@299c282e
[junit4:junit4]   2> 18132 T79 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 18133 T164 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:29524. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 18134 T164 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:29524, initiating session
[junit4:junit4]   2> 18134 T81 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:28359
[junit4:junit4]   2> 18135 T81 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:28359
[junit4:junit4]   2> 18137 T83 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca5860697000b with negotiated timeout 20000 for client /140.211.11.196:28359
[junit4:junit4]   2> 18137 T164 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:29524, sessionid = 0x13ca5860697000b, negotiated timeout = 20000
[junit4:junit4]   2> 18138 T165 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@299c282e name:ZooKeeperConnection Watcher:127.0.0.1:29524 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 18138 T79 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 18140 T84 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca5860697000b
[junit4:junit4]   2> 18155 T165 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 18154 T79 oaz.ZooKeeper.close Session: 0x13ca5860697000b closed
[junit4:junit4]   2> 18155 T81 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:28359 which had sessionid 0x13ca5860697000b
[junit4:junit4]   2> 18155 T79 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 18168 T79 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:29524/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@25f2e25b
[junit4:junit4]   2> 18169 T79 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 18170 T166 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:29524. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 18170 T166 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:29524, initiating session
[junit4:junit4]   2> 18170 T81 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:64502
[junit4:junit4]   2> 18171 T81 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:64502
[junit4:junit4]   2> 18173 T83 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca5860697000c with negotiated timeout 20000 for client /140.211.11.196:64502
[junit4:junit4]   2> 18173 T166 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:29524, sessionid = 0x13ca5860697000c, negotiated timeout = 20000
[junit4:junit4]   2> 18173 T167 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@25f2e25b name:ZooKeeperConnection Watcher:127.0.0.1:29524/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 18174 T79 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 18175 T84 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca5860697000c type:create cxid:0x1 zxid:0xa2 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 18177 T84 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca5860697000c type:create cxid:0x2 zxid:0xa3 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 18184 T79 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48723_gd_w%2Fgh
[junit4:junit4]   2> 18185 T84 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca5860697000c type:delete cxid:0x4 zxid:0xa4 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:48723_gd_w%2Fgh Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:48723_gd_w%2Fgh
[junit4:junit4]   2> 18187 T79 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:48723_gd_w%2Fgh
[junit4:junit4]   2> 18191 T121 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 18191 T102 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 18192 T136 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 18192 T108 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 18192 T151 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 18193 T136 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18193 T151 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18193 T108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18195 T121 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 18195 T102 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 18197 T79 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 18206 T168 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359986707385/collection1
[junit4:junit4]   2> 18207 T168 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 18208 T168 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 18208 T168 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 18210 T168 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359986707385/collection1/'
[junit4:junit4]   2> 18212 T168 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359986707385/collection1/lib/README' to classloader
[junit4:junit4]   2> 18213 T168 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359986707385/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 18338 T168 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 18437 T168 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 18502 T103 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 18503 T103 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:26927_gd_w%2Fgh",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:26927/gd_w/gh"}
[junit4:junit4]   2> 18522 T151 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18522 T136 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18522 T108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18522 T121 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18522 T102 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18538 T168 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 18538 T167 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18582 T168 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 19649 T168 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 19661 T168 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 19665 T168 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 19683 T168 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 19699 T168 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 19704 T168 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 19706 T168 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 19708 T168 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359986707385/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty4/
[junit4:junit4]   2> 19708 T168 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@36325789
[junit4:junit4]   2> 19709 T168 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 19711 T168 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty4 forceNew:false
[junit4:junit4]   2> 19712 T168 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty4
[junit4:junit4]   2> 19712 T168 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty4/index/
[junit4:junit4]   2> 19713 T168 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 19714 T168 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty4/index forceNew:false
[junit4:junit4]   2> 19719 T168 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3aac2343; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 19719 T168 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 19720 T168 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty4/index
[junit4:junit4]   2> 19721 T168 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 19723 T168 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 19724 T168 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 19724 T168 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 19724 T168 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 19725 T168 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 19725 T168 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 19726 T168 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 19727 T168 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 19728 T168 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 19728 T168 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 19729 T168 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 19730 T168 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 19731 T168 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 19731 T168 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 19731 T168 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 19732 T168 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 19733 T168 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 19734 T168 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 19735 T168 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 19736 T168 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 19737 T168 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 19737 T168 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 19738 T168 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 19739 T168 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 19740 T168 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 19741 T168 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 19752 T168 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 19756 T168 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 19758 T168 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 19760 T168 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty4
[junit4:junit4]   2> 19761 T168 oass.SolrIndexSearcher.<init> Opening Searcher@4cabd349 main
[junit4:junit4]   2> 19762 T168 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 19763 T168 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 19764 T168 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 19770 T168 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 19825 T169 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4cabd349 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 19834 T168 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 19835 T168 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 20028 T103 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 20029 T103 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:48723_gd_w%2Fgh",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:48723/gd_w/gh"}
[junit4:junit4]   2> 20029 T103 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 20029 T103 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 20042 T151 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20042 T102 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20042 T108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20042 T121 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20042 T167 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20042 T136 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20837 T168 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 20837 T168 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:48723/gd_w/gh collection:collection1 shard:shard2
[junit4:junit4]   2> 20842 T168 oasc.ZkController.register We are http://127.0.0.1:48723/gd_w/gh/collection1/ and leader is http://127.0.0.1:23227/gd_w/gh/collection1/
[junit4:junit4]   2> 20842 T168 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:48723/gd_w/gh
[junit4:junit4]   2> 20843 T168 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 20843 T168 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C3 name=collection1 org.apache.solr.core.SolrCore@25bd6851 url=http://127.0.0.1:48723/gd_w/gh/collection1 node=127.0.0.1:48723_gd_w%2Fgh C3_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:48723_gd_w%2Fgh, base_url=http://127.0.0.1:48723/gd_w/gh}
[junit4:junit4]   2> 20844 T170 C3 P48723 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 20844 T168 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 20845 T170 C3 P48723 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 20845 T170 C3 P48723 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 20845 T170 C3 P48723 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 20846 T79 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 20847 T79 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 20847 T170 C3 P48723 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 20847 T79 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 20867 T79 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 20870 T79 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4]   2> 20871 T79 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 21547 T103 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 21549 T103 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:48723_gd_w%2Fgh",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:48723/gd_w/gh"}
[junit4:junit4]   2> 21566 T167 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 21566 T102 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 21566 T136 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 21566 T108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 21566 T121 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 21566 T151 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2>  C2_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:26927_gd_w%2Fgh, base_url=http://127.0.0.1:26927/gd_w/gh}
[junit4:junit4]   2> 21612 T154 C2 P26927 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:36837/gd_w/gh/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 21623 T154 C2 P26927 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 21645 T154 C2 P26927 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:26927/gd_w/gh START replicas=[http://127.0.0.1:36837/gd_w/gh/collection1/] nUpdates=100
[junit4:junit4]   2> 21661 T154 C2 P26927 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 21662 T154 C2 P26927 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 21662 T154 C2 P26927 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 21663 T154 C2 P26927 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 21663 T154 C2 P26927 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 21663 T154 C2 P26927 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:36837/gd_w/gh/collection1/. core=collection1
[junit4:junit4]   2> 21664 T154 C2 P26927 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C4 name=collection1 org.apache.solr.core.SolrCore@29301a2f url=http://127.0.0.1:36837/gd_w/gh/collection1 node=127.0.0.1:36837_gd_w%2Fgh C4_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:36837_gd_w%2Fgh, base_url=http://127.0.0.1:36837/gd_w/gh, leader=true}
[junit4:junit4]   2> 21729 T112 C4 P36837 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=7 
[junit4:junit4]   2> 21747 T113 C4 P36837 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 21748 T113 C4 P36837 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty1
[junit4:junit4]   2> 21753 T113 C4 P36837 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@396cd7c2; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 21753 T113 C4 P36837 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 21758 T113 C4 P36837 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@396cd7c2; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@396cd7c2; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 21758 T113 C4 P36837 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 21760 T113 C4 P36837 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty1
[junit4:junit4]   2> 21760 T113 C4 P36837 oass.SolrIndexSearcher.<init> Opening Searcher@2829abfd realtime
[junit4:junit4]   2> 21761 T113 C4 P36837 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 21761 T113 C4 P36837 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 39
[junit4:junit4]   2> 21827 T154 C2 P26927 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 21828 T154 C2 P26927 oascsi.HttpClientUtil.createClient Creating new http client, config:connTimeout=5000&socketTimeout=20000&allowCompression=false&maxConnections=10000&maxConnectionsPerHost=10000
[junit4:junit4]   2> 21858 T154 C2 P26927 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 21873 T79 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 21932 T114 C4 P36837 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 21933 T114 C4 P36837 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=72 
[junit4:junit4]   2> 21935 T154 C2 P26927 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 21935 T154 C2 P26927 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 21936 T154 C2 P26927 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 21940 T114 C4 P36837 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty1
[junit4:junit4]   2> 21941 T114 C4 P36837 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty1/index
[junit4:junit4]   2> 21941 T114 C4 P36837 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=2 
[junit4:junit4]   2> 21943 T154 C2 P26927 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 21945 T154 C2 P26927 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty3/index.20130204080511608 forceNew:false
[junit4:junit4]   2> 21945 T154 C2 P26927 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty3
[junit4:junit4]   2> 21946 T154 C2 P26927 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty3/index.20130204080511608 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c55252c; maxCacheMB=48.0 maxMergeSizeMB=4.0)) fullCopy=true
[junit4:junit4]   2> 21962 T114 C4 P36837 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=5 
[junit4:junit4]   2> 21978 T154 C2 P26927 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty3/index
[junit4:junit4]   2> 21978 T154 C2 P26927 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 21979 T154 C2 P26927 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130204080511608
[junit4:junit4]   2> 21986 T154 C2 P26927 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty3
[junit4:junit4]   2> 21987 T154 C2 P26927 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty3
[junit4:junit4]   2> 21988 T154 C2 P26927 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty3/index/ new=./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty3/index.20130204080511608
[junit4:junit4]   2> 21992 T154 C2 P26927 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty3/index.20130204080511608 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c55252c; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 21993 T154 C2 P26927 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 21995 T154 C2 P26927 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty3
[junit4:junit4]   2> 22000 T154 C2 P26927 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 22000 T154 C2 P26927 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 22000 T154 C2 P26927 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 22002 T154 C2 P26927 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty3/index.20130204080511608
[junit4:junit4]   2> 22003 T154 C2 P26927 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty3
[junit4:junit4]   2> 22007 T154 C2 P26927 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty3/index.20130204080511608 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c55252c; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 22007 T154 C2 P26927 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 22008 T154 C2 P26927 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 22009 T154 C2 P26927 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty3
[junit4:junit4]   2> 22009 T154 C2 P26927 oass.SolrIndexSearcher.<init> Opening Searcher@1ce00c07 main
[junit4:junit4]   2> 22013 T153 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1ce00c07 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 22014 T153 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty3/index
[junit4:junit4]   2> 22015 T154 C2 P26927 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 22018 T154 C2 P26927 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty3/index.20130204080511608 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c55252c; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty3/index.20130204080511608 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1c55252c; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 22019 T154 C2 P26927 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 22020 T154 C2 P26927 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty3
[junit4:junit4]   2> 22021 T154 C2 P26927 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 22021 T154 C2 P26927 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty3/index.20130204080511608
[junit4:junit4]   2> 22021 T154 C2 P26927 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty3/index
[junit4:junit4]   2> 22022 T154 C2 P26927 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 22022 T154 C2 P26927 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 22022 T154 C2 P26927 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 22023 T154 C2 P26927 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 22025 T154 C2 P26927 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 22877 T79 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 23073 T103 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 23075 T103 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:26927_gd_w%2Fgh",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:26927/gd_w/gh"}
[junit4:junit4]   2> 23090 T151 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 23090 T121 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 23090 T136 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 23090 T108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 23090 T102 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 23090 T167 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2>  C3_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:48723_gd_w%2Fgh, base_url=http://127.0.0.1:48723/gd_w/gh}
[junit4:junit4]   2> 23871 T170 C3 P48723 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:23227/gd_w/gh/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 23871 T170 C3 P48723 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:48723/gd_w/gh START replicas=[http://127.0.0.1:23227/gd_w/gh/collection1/] nUpdates=100
[junit4:junit4]   2> 23872 T170 C3 P48723 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 23872 T170 C3 P48723 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 23873 T170 C3 P48723 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 23873 T170 C3 P48723 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 23873 T170 C3 P48723 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 23874 T170 C3 P48723 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:23227/gd_w/gh/collection1/. core=collection1
[junit4:junit4]   2> 23875 T170 C3 P48723 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 23881 T79 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> ASYNC  NEW_CORE C5 name=collection1 org.apache.solr.core.SolrCore@3a7780f0 url=http://127.0.0.1:23227/gd_w/gh/collection1 node=127.0.0.1:23227_gd_w%2Fgh C5_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:23227_gd_w%2Fgh, base_url=http://127.0.0.1:23227/gd_w/gh, leader=true}
[junit4:junit4]   2> 23887 T127 C5 P23227 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 23898 T128 C5 P23227 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 23900 T128 C5 P23227 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty2
[junit4:junit4]   2> 23905 T128 C5 P23227 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@547b471e; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 23906 T128 C5 P23227 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 23911 T128 C5 P23227 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@547b471e; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@547b471e; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 23912 T128 C5 P23227 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 23913 T128 C5 P23227 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty2
[junit4:junit4]   2> 23914 T128 C5 P23227 oass.SolrIndexSearcher.<init> Opening Searcher@4b712592 realtime
[junit4:junit4]   2> 23915 T128 C5 P23227 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 23915 T128 C5 P23227 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 17
[junit4:junit4]   2> 23917 T170 C3 P48723 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 23918 T170 C3 P48723 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 23922 T129 C5 P23227 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 23923 T129 C5 P23227 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 23924 T170 C3 P48723 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 23924 T170 C3 P48723 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 23925 T170 C3 P48723 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 23928 T129 C5 P23227 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty2
[junit4:junit4]   2> 23929 T129 C5 P23227 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty2/index
[junit4:junit4]   2> 23930 T129 C5 P23227 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=2 
[junit4:junit4]   2> 23931 T170 C3 P48723 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 23933 T170 C3 P48723 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty4/index.20130204080513597 forceNew:false
[junit4:junit4]   2> 23934 T170 C3 P48723 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty4
[junit4:junit4]   2> 23935 T170 C3 P48723 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty4/index.20130204080513597 lockFactory=org.apache.lucene.store.NativeFSLockFactory@538f7b25; maxCacheMB=48.0 maxMergeSizeMB=4.0)) fullCopy=true
[junit4:junit4]   2> 23940 T129 C5 P23227 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> 23942 T170 C3 P48723 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty4/index
[junit4:junit4]   2> 23942 T170 C3 P48723 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 23943 T170 C3 P48723 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130204080513597
[junit4:junit4]   2> 23945 T170 C3 P48723 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty4
[junit4:junit4]   2> 23945 T170 C3 P48723 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty4
[junit4:junit4]   2> 23946 T170 C3 P48723 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty4/index/ new=./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty4/index.20130204080513597
[junit4:junit4]   2> 23950 T170 C3 P48723 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty4/index.20130204080513597 lockFactory=org.apache.lucene.store.NativeFSLockFactory@538f7b25; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 23951 T170 C3 P48723 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 23953 T170 C3 P48723 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty4
[junit4:junit4]   2> 23953 T170 C3 P48723 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 23954 T170 C3 P48723 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 23954 T170 C3 P48723 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 23955 T170 C3 P48723 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty4/index.20130204080513597
[junit4:junit4]   2> 23956 T170 C3 P48723 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty4
[junit4:junit4]   2> 23960 T170 C3 P48723 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty4/index.20130204080513597 lockFactory=org.apache.lucene.store.NativeFSLockFactory@538f7b25; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 23961 T170 C3 P48723 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 23961 T170 C3 P48723 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 23962 T170 C3 P48723 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty4
[junit4:junit4]   2> 23963 T170 C3 P48723 oass.SolrIndexSearcher.<init> Opening Searcher@781f9364 main
[junit4:junit4]   2> 23966 T169 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@781f9364 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 23967 T169 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty4/index
[junit4:junit4]   2> 23968 T170 C3 P48723 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 23972 T170 C3 P48723 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty4/index.20130204080513597 lockFactory=org.apache.lucene.store.NativeFSLockFactory@538f7b25; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty4/index.20130204080513597 lockFactory=org.apache.lucene.store.NativeFSLockFactory@538f7b25; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 23972 T170 C3 P48723 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 23974 T170 C3 P48723 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty4
[junit4:junit4]   2> 23974 T170 C3 P48723 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 23975 T170 C3 P48723 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty4/index.20130204080513597
[junit4:junit4]   2> 23975 T170 C3 P48723 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359986689673/jetty4/index
[junit4:junit4]   2> 23975 T170 C3 P48723 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 23975 T170 C3 P48723 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 23976 T170 C3 P48723 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 23976 T170 C3 P48723 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 23978 T170 C3 P48723 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 24596 T103 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 24598 T103 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:48723_gd_w%2Fgh",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:48723/gd_w/gh"}
[junit4:junit4]   2> 24618 T167 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 24619 T108 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 24619 T136 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 24618 T121 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 24618 T151 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 24618 T102 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 24884 T79 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 24885 T79 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4]   2> ASYNC  NEW_CORE C6 name=collection1 org.apache.solr.core.SolrCore@4ec065f7 url=http://127.0.0.1:44201/gd_w/gh/collection1 node=127.0.0.1:44201_gd_w%2Fgh C6_STATE=coll:control_collection core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:44201_gd_w%2Fgh, base_url=http://127.0.0.

[...truncated too long message...]

lr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:794)
[junit4:junit4]    > 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4:junit4]    > 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[junit4:junit4]    > 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4:junit4]    > 	at java.lang.reflect.Method.invoke(Method.java:601)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4:junit4]    > 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]    > 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]    > 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]    > Caused by: java.net.SocketException: Socket closed
[junit4:junit4]    > 	at java.net.SocketInputStream.socketRead0(Native Method)
[junit4:junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:150)
[junit4:junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:121)
[junit4:junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:149)
[junit4:junit4]    > 	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:111)
[junit4:junit4]    > 	at org.apache.http.impl.io.AbstractSessionInputBuffer.read(AbstractSessionInputBuffer.java:195)
[junit4:junit4]    > 	at org.apache.http.impl.io.ContentLengthInputStream.read(ContentLengthInputStream.java:176)
[junit4:junit4]    > 	at org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:138)
[junit4:junit4]    > 	at org.apache.solr.common.util.FastInputStream.readWrappedStream(FastInputStream.java:80)
[junit4:junit4]    > 	at org.apache.solr.common.util.FastInputStream.refill(FastInputStream.java:89)
[junit4:junit4]    > 	at org.apache.solr.common.util.FastInputStream.readByte(FastInputStream.java:192)
[junit4:junit4]    > 	at org.apache.solr.common.util.JavaBinCodec.unmarshal(JavaBinCodec.java:107)
[junit4:junit4]    > 	at org.apache.solr.client.solrj.impl.BinaryResponseParser.processResponse(BinaryResponseParser.java:41)
[junit4:junit4]    > 	... 48 more
[junit4:junit4]   2> 284628 T79 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4]   2> NOTE: test params are: codec=Lucene42: {n_f1=Lucene41(blocksize=128), foo_b=PostingsFormat(name=Asserting), cat=Pulsing41(freqCutoff=9 minBlockSize=41 maxBlockSize=141), foo_d=PostingsFormat(name=Memory doPackFST= true), foo_f=PostingsFormat(name=Asserting), n_tl1=Lucene41(blocksize=128), n_d1=Pulsing41(freqCutoff=9 minBlockSize=41 maxBlockSize=141), rnd_b=PostingsFormat(name=Memory doPackFST= true), intDefault=Lucene41(blocksize=128), n_td1=Lucene41(blocksize=128), timestamp=Pulsing41(freqCutoff=9 minBlockSize=41 maxBlockSize=141), id=PostingsFormat(name=Memory doPackFST= true), range_facet_sl=PostingsFormat(name=Memory doPackFST= true), range_facet_si=Pulsing41(freqCutoff=9 minBlockSize=41 maxBlockSize=141), oddField_s=PostingsFormat(name=Memory doPackFST= true), sequence_i=PostingsFormat(name=Memory doPackFST= true), name=PostingsFormat(name=Memory doPackFST= true), foo_i=Lucene41(blocksize=128), regex_dup_B_s=Pulsing41(freqCutoff=9 minBlockSize=41 maxBlockSize=141), multiDefault=Pulsing41(freqCutoff=9 minBlockSize=41 maxBlockSize=141), n_tf1=Pulsing41(freqCutoff=9 minBlockSize=41 maxBlockSize=141), n_dt1=Lucene41(blocksize=128), genre_s=PostingsFormat(name=Memory doPackFST= true), author_t=Lucene41(blocksize=128), n_ti1=PostingsFormat(name=Memory doPackFST= true), range_facet_l=Pulsing41(freqCutoff=9 minBlockSize=41 maxBlockSize=141), text=PostingsFormat(name=Asserting), _version_=Lucene41(blocksize=128), val_i=PostingsFormat(name=Memory doPackFST= true), SubjectTerms_mfacet=Lucene41(blocksize=128), series_t=Lucene41(blocksize=128), a_t=Pulsing41(freqCutoff=9 minBlockSize=41 maxBlockSize=141), n_tdt1=Lucene41(blocksize=128), regex_dup_A_s=PostingsFormat(name=Asserting), price=PostingsFormat(name=Asserting), other_tl1=Lucene41(blocksize=128), n_l1=Pulsing41(freqCutoff=9 minBlockSize=41 maxBlockSize=141), a_si=Lucene41(blocksize=128), inStock=PostingsFormat(name=Asserting)}, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=zh_HK, timezone=Canada/Saskatchewan
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=1,free=206252408,total=472907776
[junit4:junit4]   2> NOTE: All tests run in this JVM: [UpdateRequestProcessorFactoryTest, TestZkChroot, PreAnalyzedFieldTest, SampleTest, TestPerFieldSimilarity, SpellCheckComponentTest, BasicDistributedZkTest]
[junit4:junit4] Completed on J0 in 285.06s, 1 test, 1 error <<< FAILURES!

[...truncated 671 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:346: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/common-build.xml:428: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:1176: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:841: There were test failures: 249 suites, 1038 tests, 1 error, 23 ignored (5 assumptions)

Total time: 41 minutes 32 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-trunk-java7 - Build # 3706 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-java7/3706/

2 tests failed.
FAILED:  org.apache.solr.cloud.BasicDistributedZkTest.testDistribSearch

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([D57BB880CB873303]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([D57BB880CB873303]:0)




Build Log:
[...truncated 9395 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
[junit4:junit4]   2> 1 T392 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4]   2> 7 T392 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZkTest-1359964174397
[junit4:junit4]   2> 18 T392 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 26 T393 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 116 T393 oaz.Environment.logEnv Server environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT
[junit4:junit4]   2> 116 T393 oaz.Environment.logEnv Server environment:host.name=lucene.zones.apache.org
[junit4:junit4]   2> 116 T393 oaz.Environment.logEnv Server environment:java.version=1.7.0_09
[junit4:junit4]   2> 116 T393 oaz.Environment.logEnv Server environment:java.vendor=Oracle Corporation
[junit4:junit4]   2> 116 T393 oaz.Environment.logEnv Server environment:java.home=/usr/local/openjdk7/jre
[junit4:junit4]   2> 116 T393 oaz.Environment.logEnv Server environment:java.class.path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/classes/test:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/codecs/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-solrj/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/analysis/common/lucene-analyzers-common-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/highlighter/lucene-highlighter-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/memory/lucene-memory-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/misc/lucene-misc-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/spatial/lucene-spatial-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/suggest/lucene-suggest-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/grouping/lucene-grouping-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/queries/lucene-queries-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/queryparser/lucene-queryparser-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/cglib-nodep-2.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/commons-cli-1.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/commons-codec-1.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/commons-fileupload-1.2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/commons-lang-2.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/easymock-3.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/guava-13.0.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/javax.servlet-api-3.0.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/objenesis-1.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/spatial4j-0.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/commons-io-2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/httpclient-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/httpcore-4.1.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/httpmime-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/jcl-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/slf4j-api-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/slf4j-jdk14-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/wstx-asl-3.2.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/zookeeper-3.4.5.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-continuation-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-deploy-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-http-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-io-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-jmx-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-security-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-server-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-servlet-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-util-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-webapp-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-xml-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/servlet-api-3.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/example-DIH/solr/db/lib/derby-10.9.1.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/test-framework/lib/junit-4.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/test-framework/lib/randomizedtesting-runner-2.0.8.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/home/hudson/.ant/lib/ivy-2.2.0.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/usr/local/openjdk7/lib/tools.jar:/home/hudson/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.0.8.jar
[junit4:junit4]   2> 118 T393 oaz.Environment.logEnv Server environment:java.library.path=/usr/local/openjdk7/jre/lib/amd64/server:/usr/local/openjdk7/jre/lib/amd64:/usr/local/openjdk7/jre/../lib/amd64:/usr/local/openjdk6/jre/lib/amd64/server:/usr/local/openjdk6/jre/lib/amd64:/usr/local/openjdk6/jre/../lib/amd64:/usr/java/packages/lib/amd64:/lib:/usr/lib:/usr/local/lib
[junit4:junit4]   2> 118 T393 oaz.Environment.logEnv Server environment:java.io.tmpdir=.
[junit4:junit4]   2> 118 T393 oaz.Environment.logEnv Server environment:java.compiler=<NA>
[junit4:junit4]   2> 118 T393 oaz.Environment.logEnv Server environment:os.name=FreeBSD
[junit4:junit4]   2> 119 T393 oaz.Environment.logEnv Server environment:os.arch=amd64
[junit4:junit4]   2> 119 T393 oaz.Environment.logEnv Server environment:os.version=9.0-RELEASE
[junit4:junit4]   2> 119 T393 oaz.Environment.logEnv Server environment:user.name=hudson
[junit4:junit4]   2> 119 T393 oaz.Environment.logEnv Server environment:user.home=/home/hudson
[junit4:junit4]   2> 119 T393 oaz.Environment.logEnv Server environment:user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 170 T393 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 170 T393 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4]   2> 170 T393 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4]   2> 178 T393 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 227 T392 oasc.ZkTestServer.run start zk server on port:57486
[junit4:junit4]   2> 287 T393 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4]   2> 354 T392 oaz.Environment.logEnv Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT
[junit4:junit4]   2> 355 T392 oaz.Environment.logEnv Client environment:host.name=lucene.zones.apache.org
[junit4:junit4]   2> 355 T392 oaz.Environment.logEnv Client environment:java.version=1.7.0_09
[junit4:junit4]   2> 355 T392 oaz.Environment.logEnv Client environment:java.vendor=Oracle Corporation
[junit4:junit4]   2> 356 T392 oaz.Environment.logEnv Client environment:java.home=/usr/local/openjdk7/jre
[junit4:junit4]   2> 356 T392 oaz.Environment.logEnv Client environment:java.class.path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/classes/test:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/codecs/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-solrj/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/analysis/common/lucene-analyzers-common-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/highlighter/lucene-highlighter-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/memory/lucene-memory-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/misc/lucene-misc-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/spatial/lucene-spatial-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/suggest/lucene-suggest-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/grouping/lucene-grouping-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/queries/lucene-queries-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/queryparser/lucene-queryparser-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/cglib-nodep-2.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/commons-cli-1.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/commons-codec-1.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/commons-fileupload-1.2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/commons-lang-2.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/easymock-3.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/guava-13.0.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/javax.servlet-api-3.0.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/objenesis-1.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/spatial4j-0.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/commons-io-2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/httpclient-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/httpcore-4.1.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/httpmime-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/jcl-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/slf4j-api-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/slf4j-jdk14-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/wstx-asl-3.2.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/zookeeper-3.4.5.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-continuation-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-deploy-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-http-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-io-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-jmx-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-security-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-server-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-servlet-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-util-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-webapp-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-xml-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/servlet-api-3.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/example-DIH/solr/db/lib/derby-10.9.1.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/test-framework/lib/junit-4.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/test-framework/lib/randomizedtesting-runner-2.0.8.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/home/hudson/.ant/lib/ivy-2.2.0.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/usr/local/openjdk7/lib/tools.jar:/home/hudson/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.0.8.jar
[junit4:junit4]   2> 358 T392 oaz.Environment.logEnv Client environment:java.library.path=/usr/local/openjdk7/jre/lib/amd64/server:/usr/local/openjdk7/jre/lib/amd64:/usr/local/openjdk7/jre/../lib/amd64:/usr/local/openjdk6/jre/lib/amd64/server:/usr/local/openjdk6/jre/lib/amd64:/usr/local/openjdk6/jre/../lib/amd64:/usr/java/packages/lib/amd64:/lib:/usr/lib:/usr/local/lib
[junit4:junit4]   2> 359 T392 oaz.Environment.logEnv Client environment:java.io.tmpdir=.
[junit4:junit4]   2> 359 T392 oaz.Environment.logEnv Client environment:java.compiler=<NA>
[junit4:junit4]   2> 359 T392 oaz.Environment.logEnv Client environment:os.name=FreeBSD
[junit4:junit4]   2> 360 T392 oaz.Environment.logEnv Client environment:os.arch=amd64
[junit4:junit4]   2> 360 T392 oaz.Environment.logEnv Client environment:os.version=9.0-RELEASE
[junit4:junit4]   2> 361 T392 oaz.Environment.logEnv Client environment:user.name=hudson
[junit4:junit4]   2> 361 T392 oaz.Environment.logEnv Client environment:user.home=/home/hudson
[junit4:junit4]   2> 361 T392 oaz.Environment.logEnv Client environment:user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 368 T392 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57486 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@236a9eb8
[junit4:junit4]   2> 477 T392 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 490 T398 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:57486. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 493 T398 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57486, initiating session
[junit4:junit4]   2> 493 T394 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:57487
[junit4:junit4]   2> 527 T394 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:57487
[junit4:junit4]   2> 545 T396 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 587 T396 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca42e798a0000 with negotiated timeout 10000 for client /140.211.11.196:57487
[junit4:junit4]   2> 588 T398 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:57486, sessionid = 0x13ca42e798a0000, negotiated timeout = 10000
[junit4:junit4]   2> 596 T399 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@236a9eb8 name:ZooKeeperConnection Watcher:127.0.0.1:57486 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 596 T392 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 602 T392 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 688 T397 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca42e798a0000
[junit4:junit4]   2> 690 T399 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 690 T392 oaz.ZooKeeper.close Session: 0x13ca42e798a0000 closed
[junit4:junit4]   2> 690 T392 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57486/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@79399317
[junit4:junit4]   2> 691 T394 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:57487 which had sessionid 0x13ca42e798a0000
[junit4:junit4]   2> 691 T392 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 692 T400 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:57486. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 693 T400 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57486, initiating session
[junit4:junit4]   2> 693 T394 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:57488
[junit4:junit4]   2> 693 T394 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:57488
[junit4:junit4]   2> 695 T396 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca42e798a0001 with negotiated timeout 10000 for client /140.211.11.196:57488
[junit4:junit4]   2> 695 T400 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:57486, sessionid = 0x13ca42e798a0001, negotiated timeout = 10000
[junit4:junit4]   2> 695 T401 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@79399317 name:ZooKeeperConnection Watcher:127.0.0.1:57486/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 696 T392 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 703 T392 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 710 T392 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 715 T392 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 719 T392 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 725 T392 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 728 T392 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 740 T392 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 743 T392 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 850 T392 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 851 T392 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 856 T392 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 857 T392 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 862 T392 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 863 T392 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 868 T392 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 869 T392 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 874 T392 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 875 T392 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 880 T392 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 881 T392 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 886 T392 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 887 T392 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 892 T397 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca42e798a0001
[junit4:junit4]   2> 893 T401 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 893 T392 oaz.ZooKeeper.close Session: 0x13ca42e798a0001 closed
[junit4:junit4]   2> 893 T394 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:57488 which had sessionid 0x13ca42e798a0001
[junit4:junit4]   2> 1085 T392 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 1097 T392 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:57489
[junit4:junit4]   2> 1098 T392 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 1098 T392 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 1099 T392 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359964175283
[junit4:junit4]   2> 1100 T392 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359964175283/solr.xml
[junit4:junit4]   2> 1101 T392 oasc.CoreContainer.<init> New CoreContainer 281542132
[junit4:junit4]   2> 1102 T392 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359964175283/'
[junit4:junit4]   2> 1102 T392 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359964175283/'
[junit4:junit4]   2> 1165 T392 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 1166 T392 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 1166 T392 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 1167 T392 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 1167 T392 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 1167 T392 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 1168 T392 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 1168 T392 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 1169 T392 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 1169 T392 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 1195 T392 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 1219 T392 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:57486/solr
[junit4:junit4]   2> 1250 T392 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 1250 T392 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57486 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@6f255853
[junit4:junit4]   2> 1251 T392 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1252 T411 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:57486. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 1253 T411 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57486, initiating session
[junit4:junit4]   2> 1253 T394 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:57490
[junit4:junit4]   2> 1254 T394 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:57490
[junit4:junit4]   2> 1255 T396 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca42e798a0002 with negotiated timeout 20000 for client /140.211.11.196:57490
[junit4:junit4]   2> 1255 T411 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:57486, sessionid = 0x13ca42e798a0002, negotiated timeout = 20000
[junit4:junit4]   2> 1256 T412 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6f255853 name:ZooKeeperConnection Watcher:127.0.0.1:57486 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1256 T392 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1258 T397 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca42e798a0002
[junit4:junit4]   2> 1259 T412 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 1259 T392 oaz.ZooKeeper.close Session: 0x13ca42e798a0002 closed
[junit4:junit4]   2> 1260 T394 oazs.NIOServerCnxn.doIO WARNING caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x13ca42e798a0002, likely client has closed socket
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4:junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4:junit4]   2> 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 
[junit4:junit4]   2> 1261 T394 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:57490 which had sessionid 0x13ca42e798a0002
[junit4:junit4]   2> 1265 T392 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 1288 T392 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57486/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@36e1d274
[junit4:junit4]   2> 1289 T392 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1290 T413 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:57486. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 1291 T413 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57486, initiating session
[junit4:junit4]   2> 1291 T394 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:57491
[junit4:junit4]   2> 1291 T394 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:57491
[junit4:junit4]   2> 1319 T396 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca42e798a0003 with negotiated timeout 20000 for client /140.211.11.196:57491
[junit4:junit4]   2> 1319 T413 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:57486, sessionid = 0x13ca42e798a0003, negotiated timeout = 20000
[junit4:junit4]   2> 1320 T414 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@36e1d274 name:ZooKeeperConnection Watcher:127.0.0.1:57486/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1320 T392 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1357 T397 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca42e798a0003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 1368 T392 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 1372 T392 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57489_
[junit4:junit4]   2> 1378 T397 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca42e798a0003 type:delete cxid:0x7 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:57489_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:57489_
[junit4:junit4]   2> 1426 T392 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57489_
[junit4:junit4]   2> 1432 T392 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 1507 T397 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca42e798a0003 type:delete cxid:0x15 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 1509 T392 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 1544 T392 oasc.Overseer.start Overseer (id=89126612155105283-127.0.0.1:57489_-n_0000000000) starting
[junit4:junit4]   2> 1545 T397 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca42e798a0003 type:create cxid:0x1a zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 1568 T397 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca42e798a0003 type:create cxid:0x1b zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 1570 T397 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca42e798a0003 type:create cxid:0x1c zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 1580 T397 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca42e798a0003 type:create cxid:0x1d zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 1588 T416 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 1589 T392 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 1593 T392 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 1615 T415 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 1626 T417 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359964175283/collection1
[junit4:junit4]   2> 1626 T417 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 1627 T417 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 1628 T417 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 1630 T417 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359964175283/collection1/'
[junit4:junit4]   2> 1632 T417 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359964175283/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 1632 T417 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359964175283/collection1/lib/README' to classloader
[junit4:junit4]   2> 1719 T417 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1804 T417 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1906 T417 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 1913 T417 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 2666 T417 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 2679 T417 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 2683 T417 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 2706 T417 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2712 T417 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2718 T417 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2720 T417 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2722 T417 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359964175283/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/control/data/
[junit4:junit4]   2> 2722 T417 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c1a7261
[junit4:junit4]   2> 2723 T417 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 2763 T417 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/control/data forceNew:false
[junit4:junit4]   2> 2763 T417 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/control/data
[junit4:junit4]   2> 2764 T417 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/control/data/index/
[junit4:junit4]   2> 2765 T417 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 2766 T417 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/control/data/index forceNew:false
[junit4:junit4]   2> 2788 T417 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@46615a78),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 2789 T417 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 2790 T417 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/control/data/index
[junit4:junit4]   2> 2790 T417 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 2792 T417 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 2792 T417 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 2792 T417 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 2793 T417 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 2793 T417 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 2794 T417 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 2794 T417 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 2794 T417 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 2795 T417 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 2795 T417 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 2796 T417 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 2796 T417 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 2797 T417 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 2797 T417 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 2797 T417 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 2798 T417 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 2798 T417 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2799 T417 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2799 T417 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2799 T417 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2800 T417 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2800 T417 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2801 T417 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2801 T417 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2802 T417 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 2802 T417 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 2818 T417 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 2823 T417 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 2825 T417 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 2828 T417 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/control/data
[junit4:junit4]   2> 2830 T417 oass.SolrIndexSearcher.<init> Opening Searcher@46183eec main
[junit4:junit4]   2> 2831 T417 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 2831 T417 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 2832 T417 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 2840 T417 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 2899 T418 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@46183eec main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 2904 T417 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 2904 T417 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 2905 T397 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca42e798a0003 type:create cxid:0x50 zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 3126 T415 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3128 T415 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57489_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57489"}
[junit4:junit4]   2> 3128 T415 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4]   2> 3158 T415 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 3160 T397 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca42e798a0003 type:create cxid:0x58 zxid:0x2c txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 3204 T414 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 3918 T417 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 3918 T417 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:57489 collection:control_collection shard:shard1
[junit4:junit4]   2> 3928 T417 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 3949 T417 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4]   2> 4002 T397 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca42e798a0003 type:delete cxid:0x72 zxid:0x36 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4]   2> 4003 T417 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 4004 T397 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca42e798a0003 type:create cxid:0x73 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4015 T417 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 4015 T417 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 4016 T417 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:57489/collection1/
[junit4:junit4]   2> 4016 T417 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 4017 T417 oasc.SyncStrategy.syncToMe http://127.0.0.1:57489/collection1/ has no replicas
[junit4:junit4]   2> 4017 T417 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:57489/collection1/
[junit4:junit4]   2> 4017 T417 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4]   2> 4030 T397 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca42e798a0003 type:create cxid:0x7d zxid:0x3b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4715 T415 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 4772 T414 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 4809 T417 oasc.ZkController.register We are http://127.0.0.1:57489/collection1/ and leader is http://127.0.0.1:57489/collection1/
[junit4:junit4]   2> 4809 T417 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57489
[junit4:junit4]   2> 4810 T417 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 4810 T417 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 4810 T417 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 4814 T417 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 4816 T392 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 4816 T392 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 4817 T392 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 4830 T392 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 4848 T392 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57486/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@418e4ed7
[junit4:junit4]   2> 4850 T392 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4850 T419 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:57486. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4851 T419 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57486, initiating session
[junit4:junit4]   2> 4852 T394 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:57492
[junit4:junit4]   2> 4852 T394 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:57492
[junit4:junit4]   2> 4854 T396 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca42e798a0004 with negotiated timeout 10000 for client /140.211.11.196:57492
[junit4:junit4]   2> 4854 T419 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:57486, sessionid = 0x13ca42e798a0004, negotiated timeout = 10000
[junit4:junit4]   2> 4855 T420 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@418e4ed7 name:ZooKeeperConnection Watcher:127.0.0.1:57486/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4855 T392 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4857 T392 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 4863 T392 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 5076 T392 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 5079 T392 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:57493
[junit4:junit4]   2> 5079 T392 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 5080 T392 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 5081 T392 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359964179253
[junit4:junit4]   2> 5081 T392 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359964179253/solr.xml
[junit4:junit4]   2> 5082 T392 oasc.CoreContainer.<init> New CoreContainer 2016950128
[junit4:junit4]   2> 5082 T392 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359964179253/'
[junit4:junit4]   2> 5083 T392 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359964179253/'
[junit4:junit4]   2> 5165 T392 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 5166 T392 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 5166 T392 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 5167 T392 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 5167 T392 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 5168 T392 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 5168 T392 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 5169 T392 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 5169 T392 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 5170 T392 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 5198 T392 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 5223 T392 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:57486/solr
[junit4:junit4]   2> 5223 T392 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 5224 T392 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57486 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@54fc0a39
[junit4:junit4]   2> 5225 T392 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 5226 T430 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:57486. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 5227 T430 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57486, initiating session
[junit4:junit4]   2> 5227 T394 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:57494
[junit4:junit4]   2> 5227 T394 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:57494
[junit4:junit4]   2> 5230 T396 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca42e798a0005 with negotiated timeout 20000 for client /140.211.11.196:57494
[junit4:junit4]   2> 5230 T430 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:57486, sessionid = 0x13ca42e798a0005, negotiated timeout = 20000
[junit4:junit4]   2> 5231 T431 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@54fc0a39 name:ZooKeeperConnection Watcher:127.0.0.1:57486 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 5231 T392 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 5233 T397 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca42e798a0005
[junit4:junit4]   2> 5234 T431 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 5234 T394 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:57494 which had sessionid 0x13ca42e798a0005
[junit4:junit4]   2> 5234 T392 oaz.ZooKeeper.close Session: 0x13ca42e798a0005 closed
[junit4:junit4]   2> 5235 T392 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 5249 T392 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57486/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@5a3bf049
[junit4:junit4]   2> 5251 T392 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 5252 T432 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:57486. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 5253 T432 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57486, initiating session
[junit4:junit4]   2> 5253 T394 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:57495
[junit4:junit4]   2> 5254 T394 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:57495
[junit4:junit4]   2> 5255 T396 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca42e798a0006 with negotiated timeout 20000 for client /140.211.11.196:57495
[junit4:junit4]   2> 5256 T432 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:57486, sessionid = 0x13ca42e798a0006, negotiated timeout = 20000
[junit4:junit4]   2> 5256 T433 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5a3bf049 name:ZooKeeperConnection Watcher:127.0.0.1:57486/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 5256 T392 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 5258 T397 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca42e798a0006 type:create cxid:0x1 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5260 T397 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca42e798a0006 type:create cxid:0x2 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 5262 T392 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57493_
[junit4:junit4]   2> 5263 T397 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca42e798a0006 type:delete cxid:0x4 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:57493_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:57493_
[junit4:junit4]   2> 5270 T392 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57493_
[junit4:junit4]   2> 5298 T414 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 5298 T420 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 5299 T414 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5314 T392 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 5330 T434 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359964179253/collection1
[junit4:junit4]   2> 5331 T434 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 5332 T434 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 5333 T434 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 5336 T434 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359964179253/collection1/'
[junit4:junit4]   2> 5338 T434 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359964179253/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 5339 T434 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359964179253/collection1/lib/README' to classloader
[junit4:junit4]   2> 5432 T434 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 5521 T434 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 5623 T434 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 5630 T434 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 6294 T415 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6295 T415 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57489_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57489"}
[junit4:junit4]   2> 6360 T434 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 6363 T414 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6363 T420 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6363 T433 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6376 T434 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 6380 T434 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 6425 T434 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 6432 T434 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 6437 T434 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 6440 T434 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 6442 T434 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359964179253/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty1/
[junit4:junit4]   2> 6443 T434 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c1a7261
[junit4:junit4]   2> 6443 T434 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 6444 T434 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty1 forceNew:false
[junit4:junit4]   2> 6445 T434 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty1
[junit4:junit4]   2> 6445 T434 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty1/index/
[junit4:junit4]   2> 6446 T434 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 6446 T434 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty1/index forceNew:false
[junit4:junit4]   2> 6472 T434 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b20f25),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 6472 T434 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 6473 T434 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty1/index
[junit4:junit4]   2> 6473 T434 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 6476 T434 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 6476 T434 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 6476 T434 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 6477 T434 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 6477 T434 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 6477 T434 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 6478 T434 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 6478 T434 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 6479 T434 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 6480 T434 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 6480 T434 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 6481 T434 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 6481 T434 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 6481 T434 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 6482 T434 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 6482 T434 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 6483 T434 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6484 T434 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6484 T434 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6485 T434 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6485 T434 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6486 T434 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6486 T434 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6487 T434 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 6487 T434 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 6488 T434 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 6505 T434 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 6512 T434 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 6515 T434 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 6518 T434 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty1
[junit4:junit4]   2> 6520 T434 oass.SolrIndexSearcher.<init> Opening Searcher@45249b94 main
[junit4:junit4]   2> 6521 T434 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 6521 T434 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 6522 T434 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 6581 T434 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 6733 T435 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@45249b94 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 6738 T434 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 6739 T434 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 7909 T415 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 7910 T415 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57493_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57493"}
[junit4:junit4]   2> 7911 T415 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4]   2> 7911 T415 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 7965 T420 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7965 T414 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7965 T433 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8743 T434 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 8743 T434 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:57493 collection:collection1 shard:shard1
[junit4:junit4]   2> 8745 T434 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 8768 T397 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca42e798a0006 type:delete cxid:0x45 zxid:0x5b txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 8769 T434 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 8770 T397 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca42e798a0006 type:create cxid:0x46 zxid:0x5c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8779 T434 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 8779 T434 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 8780 T434 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:57493/collection1/
[junit4:junit4]   2> 8780 T434 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 8780 T434 oasc.SyncStrategy.syncToMe http://127.0.0.1:57493/collection1/ has no replicas
[junit4:junit4]   2> 8780 T434 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:57493/collection1/
[junit4:junit4]   2> 8781 T434 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 8788 T397 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca42e798a0006 type:create cxid:0x50 zxid:0x60 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9477 T415 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9499 T414 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 9499 T420 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 9499 T433 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 9505 T434 oasc.ZkController.register We are http://127.0.0.1:57493/collection1/ and leader is http://127.0.0.1:57493/collection1/
[junit4:junit4]   2> 9505 T434 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57493
[junit4:junit4]   2> 9505 T434 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 9506 T434 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 9506 T434 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 9515 T434 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9517 T392 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 9518 T392 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 9519 T392 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 9764 T392 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 9767 T392 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:57496
[junit4:junit4]   2> 9768 T392 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 9769 T392 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 9770 T392 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359964183926
[junit4:junit4]   2> 9770 T392 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359964183926/solr.xml
[junit4:junit4]   2> 9770 T392 oasc.CoreContainer.<init> New CoreContainer 1572554169
[junit4:junit4]   2> 9771 T392 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359964183926/'
[junit4:junit4]   2> 9772 T392 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359964183926/'
[junit4:junit4]   2> 9844 T392 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 9845 T392 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 9846 T392 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 9846 T392 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 9847 T392 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 9847 T392 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 9847 T392 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 9848 T392 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 9848 T392 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 9849 T392 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 9878 T392 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 9904 T392 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:57486/solr
[junit4:junit4]   2> 9905 T392 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 9906 T392 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57486 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@68f8529f
[junit4:junit4]   2> 9907 T392 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 9907 T445 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:57486. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 9908 T445 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57486, initiating session
[junit4:junit4]   2> 9909 T394 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:57497
[junit4:junit4]   2> 9909 T394 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:57497
[junit4:junit4]   2> 9912 T396 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca42e798a0007 with negotiated timeout 20000 for client /140.211.11.196:57497
[junit4:junit4]   2> 9912 T445 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:57486, sessionid = 0x13ca42e798a0007, negotiated timeout = 20000
[junit4:junit4]   2> 9912 T446 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@68f8529f name:ZooKeeperConnection Watcher:127.0.0.1:57486 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 9913 T392 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 9915 T397 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca42e798a0007
[junit4:junit4]   2> 9917 T446 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 9917 T394 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:57497 which had sessionid 0x13ca42e798a0007
[junit4:junit4]   2> 9917 T392 oaz.ZooKeeper.close Session: 0x13ca42e798a0007 closed
[junit4:junit4]   2> 9918 T392 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 9932 T392 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57486/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@749bfdca
[junit4:junit4]   2> 9933 T392 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 9934 T447 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:57486. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 9935 T447 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57486, initiating session
[junit4:junit4]   2> 9935 T394 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:57498
[junit4:junit4]   2> 9935 T394 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:57498
[junit4:junit4]   2> 9937 T396 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca42e798a0008 with negotiated timeout 20000 for client /140.211.11.196:57498
[junit4:junit4]   2> 9937 T447 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:57486, sessionid = 0x13ca42e798a0008, negotiated timeout = 20000
[junit4:junit4]   2> 9938 T448 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@749bfdca name:ZooKeeperConnection Watcher:127.0.0.1:57486/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 9938 T392 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 9939 T397 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca42e798a0008 type:create cxid:0x1 zxid:0x6d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9941 T397 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca42e798a0008 type:create cxid:0x2 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9944 T392 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57496_
[junit4:junit4]   2> 9945 T397 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca42e798a0008 type:delete cxid:0x4 zxid:0x6f txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:57496_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:57496_
[junit4:junit4]   2> 9951 T392 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57496_
[junit4:junit4]   2> 9955 T420 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 9956 T433 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 9956 T414 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 9957 T414 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9959 T420 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 9960 T433 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 9965 T392 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 9975 T449 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359964183926/collection1
[junit4:junit4]   2> 9975 T449 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 9977 T449 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 9977 T449 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 9979 T449 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359964183926/collection1/'
[junit4:junit4]   2> 9981 T449 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359964183926/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 9981 T449 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359964183926/collection1/lib/README' to classloader
[junit4:junit4]   2> 10045 T449 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 10121 T449 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 10222 T449 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 10232 T449 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 10982 T449 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 10997 T449 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 11003 T449 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 11019 T415 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11022 T415 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57493_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57493"}
[junit4:junit4]   2> 11041 T420 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11041 T433 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11041 T448 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11041 T414 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11048 T449 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 11055 T449 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 11062 T449 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 11064 T449 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 11067 T449 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359964183926/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty2/
[junit4:junit4]   2> 11067 T449 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c1a7261
[junit4:junit4]   2> 11068 T449 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 11069 T449 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty2 forceNew:false
[junit4:junit4]   2> 11070 T449 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty2
[junit4:junit4]   2> 11070 T449 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty2/index/
[junit4:junit4]   2> 11070 T449 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 11071 T449 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty2/index forceNew:false
[junit4:junit4]   2> 11077 T449 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@64eb9eaf),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 11078 T449 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 11079 T449 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty2/index
[junit4:junit4]   2> 11079 T449 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 11082 T449 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 11082 T449 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 11082 T449 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 11083 T449 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 11084 T449 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 11084 T449 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 11084 T449 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 11085 T449 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 11086 T449 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 11087 T449 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 11087 T449 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 11088 T449 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 11089 T449 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 11089 T449 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 11090 T449 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 11090 T449 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 11091 T449 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11092 T449 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11092 T449 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11093 T449 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11094 T449 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11094 T449 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11095 T449 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11096 T449 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11096 T449 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 11097 T449 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 11111 T449 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 11118 T449 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 11120 T449 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 11124 T449 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty2
[junit4:junit4]   2> 11126 T449 oass.SolrIndexSearcher.<init> Opening Searcher@14647a74 main
[junit4:junit4]   2> 11127 T449 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 11127 T449 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 11128 T449 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 11139 T449 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 11202 T450 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@14647a74 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 11210 T449 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 11210 T449 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 12552 T415 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12553 T415 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57496_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57496"}
[junit4:junit4]   2> 12554 T415 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 12554 T415 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 12565 T420 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12565 T414 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12565 T448 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12565 T433 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13214 T449 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 13214 T449 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:57496 collection:collection1 shard:shard2
[junit4:junit4]   2> 13216 T449 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4]   2> 13238 T397 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca42e798a0008 type:delete cxid:0x44 zxid:0x7e txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
[junit4:junit4]   2> 13240 T449 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 13241 T397 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca42e798a0008 type:create cxid:0x45 zxid:0x7f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 13246 T449 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 13246 T449 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 13247 T449 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:57496/collection1/
[junit4:junit4]   2> 13247 T449 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 13247 T449 oasc.SyncStrategy.syncToMe http://127.0.0.1:57496/collection1/ has no replicas
[junit4:junit4]   2> 13248 T449 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:57496/collection1/
[junit4:junit4]   2> 13248 T449 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4]   2> 13254 T397 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca42e798a0008 type:create cxid:0x4e zxid:0x82 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 14081 T415 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14111 T420 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 14113 T414 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 14114 T433 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 14114 T448 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 14131 T449 oasc.ZkController.register We are http://127.0.0.1:57496/collection1/ and leader is http://127.0.0.1:57496/collection1/
[junit4:junit4]   2> 14131 T449 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57496
[junit4:junit4]   2> 14131 T449 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 14132 T449 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 14132 T449 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 14135 T449 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14137 T392 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 14137 T392 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 14138 T392 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 14358 T392 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 14362 T392 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:57499
[junit4:junit4]   2> 14363 T392 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 14363 T392 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 14364 T392 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359964188544
[junit4:junit4]   2> 14364 T392 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359964188544/solr.xml
[junit4:junit4]   2> 14364 T392 oasc.CoreContainer.<init> New CoreContainer 1347951050
[junit4:junit4]   2> 14365 T392 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359964188544/'
[junit4:junit4]   2> 14366 T392 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359964188544/'
[junit4:junit4]   2> 14436 T392 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 14436 T392 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 14437 T392 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 14437 T392 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 14438 T392 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 14438 T392 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 14439 T392 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 14439 T392 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 14440 T392 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 14440 T392 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 14463 T392 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 14487 T392 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:57486/solr
[junit4:junit4]   2> 14487 T392 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 14488 T392 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57486 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@17dc56b4
[junit4:junit4]   2> 14489 T392 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 14490 T460 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:57486. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 14491 T460 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57486, initiating session
[junit4:junit4]   2> 14491 T394 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:57500
[junit4:junit4]   2> 14491 T394 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:57500
[junit4:junit4]   2> 14494 T396 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca42e798a0009 with negotiated timeout 20000 for client /140.211.11.196:57500
[junit4:junit4]   2> 14494 T460 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:57486, sessionid = 0x13ca42e798a0009, negotiated timeout = 20000
[junit4:junit4]   2> 14494 T461 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@17dc56b4 name:ZooKeeperConnection Watcher:127.0.0.1:57486 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 14494 T392 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 14497 T397 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca42e798a0009
[junit4:junit4]   2> 14510 T461 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 14510 T394 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:57500 which had sessionid 0x13ca42e798a0009
[junit4:junit4]   2> 14510 T392 oaz.ZooKeeper.close Session: 0x13ca42e798a0009 closed
[junit4:junit4]   2> 14511 T392 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 14525 T392 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57486/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@7373b483
[junit4:junit4]   2> 14526 T392 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 14527 T462 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:57486. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 14527 T462 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57486, initiating session
[junit4:junit4]   2> 14528 T394 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:57501
[junit4:junit4]   2> 14528 T394 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:57501
[junit4:junit4]   2> 14530 T396 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca42e798a000a with negotiated timeout 20000 for client /140.211.11.196:57501
[junit4:junit4]   2> 14530 T462 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:57486, sessionid = 0x13ca42e798a000a, negotiated timeout = 20000
[junit4:junit4]   2> 14530 T463 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7373b483 name:ZooKeeperConnection Watcher:127.0.0.1:57486/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 14530 T392 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 14532 T397 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca42e798a000a type:create cxid:0x1 zxid:0x8f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 14535 T397 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca42e798a000a type:create cxid:0x2 zxid:0x90 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 14538 T392 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57499_
[junit4:junit4]   2> 14539 T397 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca42e798a000a type:delete cxid:0x4 zxid:0x91 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:57499_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:57499_
[junit4:junit4]   2> 14540 T392 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57499_
[junit4:junit4]   2> 14544 T420 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 14544 T433 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 14545 T448 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 14546 T448 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14546 T414 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 14546 T414 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14548 T420 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 14549 T433 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 14553 T392 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 14562 T464 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359964188544/collection1
[junit4:junit4]   2> 14563 T464 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 14564 T464 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 14564 T464 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 14568 T464 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359964188544/collection1/'
[junit4:junit4]   2> 14569 T464 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359964188544/collection1/lib/README' to classloader
[junit4:junit4]   2> 14570 T464 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359964188544/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 14627 T464 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 14721 T464 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 14823 T464 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 14830 T464 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 15440 T464 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 15449 T464 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 15452 T464 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 15490 T464 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 15499 T464 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 15507 T464 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 15510 T464 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 15513 T464 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359964188544/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty3/
[junit4:junit4]   2> 15513 T464 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c1a7261
[junit4:junit4]   2> 15514 T464 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 15515 T464 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty3 forceNew:false
[junit4:junit4]   2> 15516 T464 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty3
[junit4:junit4]   2> 15516 T464 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty3/index/
[junit4:junit4]   2> 15517 T464 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 15518 T464 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty3/index forceNew:false
[junit4:junit4]   2> 15550 T464 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6ff29305),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 15551 T464 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 15552 T464 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty3/index
[junit4:junit4]   2> 15552 T464 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 15555 T464 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 15555 T464 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 15555 T464 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 15556 T464 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 15557 T464 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 15557 T464 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 15557 T464 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 15558 T464 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 15559 T464 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 15560 T464 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 15560 T464 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 15561 T464 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 15562 T464 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 15562 T464 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 15563 T464 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 15563 T464 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 15564 T464 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15565 T464 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15565 T464 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15566 T464 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15566 T464 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15567 T464 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15568 T464 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15569 T464 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15569 T464 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 15570 T464 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 15586 T464 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 15594 T464 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 15596 T464 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 15599 T464 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty3
[junit4:junit4]   2> 15601 T464 oass.SolrIndexSearcher.<init> Opening Searcher@6a9d8d3f main
[junit4:junit4]   2> 15603 T464 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 15603 T464 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 15604 T464 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 15614 T464 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 15625 T415 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 15626 T415 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57496_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57496"}
[junit4:junit4]   2> 15646 T414 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15647 T448 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15647 T433 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15646 T463 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15646 T420 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15682 T465 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6a9d8d3f main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 15689 T464 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 15689 T464 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 17172 T415 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17173 T415 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57499_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57499"}
[junit4:junit4]   2> 17173 T415 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 17173 T415 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 17192 T448 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 17192 T433 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 17192 T414 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 17193 T463 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 17192 T420 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 17694 T464 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 17694 T464 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:57499 collection:collection1 shard:shard1
[junit4:junit4]   2> 17707 T464 oasc.ZkController.register We are http://127.0.0.1:57499/collection1/ and leader is http://127.0.0.1:57493/collection1/
[junit4:junit4]   2> 17707 T464 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57499
[junit4:junit4]   2> 17707 T464 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 17708 T464 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C66 name=collection1 org.apache.solr.core.SolrCore@329bfb6c url=http://127.0.0.1:57499/collection1 node=127.0.0.1:57499_ C66_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:57499_, base_url=http://127.0.0.1:57499}
[junit4:junit4]   2> 17715 T466 C66 P57499 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 17716 T466 C66 P57499 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 17716 T464 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17716 T466 C66 P57499 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 17716 T466 C66 P57499 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 17718 T392 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 17718 T392 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 17719 T466 C66 P57499 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 17719 T392 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 17939 T392 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 17942 T392 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:57503
[junit4:junit4]   2> 17943 T392 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 17944 T392 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 17945 T392 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359964192127
[junit4:junit4]   2> 17945 T392 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359964192127/solr.xml
[junit4:junit4]   2> 17946 T392 oasc.CoreContainer.<init> New CoreContainer 365908951
[junit4:junit4]   2> 17947 T392 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359964192127/'
[junit4:junit4]   2> 17948 T392 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359964192127/'
[junit4:junit4]   2> 18012 T392 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 18012 T392 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 18013 T392 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 18013 T392 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 18014 T392 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 18014 T392 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 18015 T392 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 18015 T392 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 18016 T392 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 18016 T392 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 18041 T392 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 18079 T392 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:57486/solr
[junit4:junit4]   2> 18080 T392 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 18081 T392 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57486 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@27835573
[junit4:junit4]   2> 18082 T392 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 18082 T476 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:57486. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 18083 T476 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57486, initiating session
[junit4:junit4]   2> 18084 T394 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:57504
[junit4:junit4]   2> 18084 T394 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:57504
[junit4:junit4]   2> 18091 T396 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca42e798a000b with negotiated timeout 20000 for client /140.211.11.196:57504
[junit4:junit4]   2> 18091 T476 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:57486, sessionid = 0x13ca42e798a000b, negotiated timeout = 20000
[junit4:junit4]   2> 18092 T477 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@27835573 name:ZooKeeperConnection Watcher:127.0.0.1:57486 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 18092 T392 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 18094 T397 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca42e798a000b
[junit4:junit4]   2> 18095 T394 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:57504 which had sessionid 0x13ca42e798a000b
[junit4:junit4]   2> 18096 T477 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 18096 T392 oaz.ZooKeeper.close Session: 0x13ca42e798a000b closed
[junit4:junit4]   2> 18096 T392 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 18112 T392 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:57486/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@15ec85e2
[junit4:junit4]   2> 18113 T392 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 18114 T478 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:57486. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 18114 T478 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:57486, initiating session
[junit4:junit4]   2> 18115 T394 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:57505
[junit4:junit4]   2> 18115 T394 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:57505
[junit4:junit4]   2> 18117 T396 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca42e798a000c with negotiated timeout 20000 for client /140.211.11.196:57505
[junit4:junit4]   2> 18117 T478 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:57486, sessionid = 0x13ca42e798a000c, negotiated timeout = 20000
[junit4:junit4]   2> 18117 T479 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@15ec85e2 name:ZooKeeperConnection Watcher:127.0.0.1:57486/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 18117 T392 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 18119 T397 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca42e798a000c type:create cxid:0x1 zxid:0xa2 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 18123 T397 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca42e798a000c type:create cxid:0x2 zxid:0xa3 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 18125 T392 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57503_
[junit4:junit4]   2> 18126 T397 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca42e798a000c type:delete cxid:0x4 zxid:0xa4 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:57503_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:57503_
[junit4:junit4]   2> 18128 T392 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57503_
[junit4:junit4]   2> 18136 T420 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 18136 T463 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 18136 T433 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 18137 T448 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 18138 T414 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 18138 T448 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18138 T414 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18140 T463 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 18141 T433 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 18141 T420 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 18146 T392 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 18157 T480 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359964192127/collection1
[junit4:junit4]   2> 18158 T480 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 18159 T480 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 18160 T480 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 18162 T480 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359964192127/collection1/'
[junit4:junit4]   2> 18164 T480 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359964192127/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 18165 T480 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359964192127/collection1/lib/README' to classloader
[junit4:junit4]   2> 18229 T480 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 18313 T480 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 18414 T480 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 18422 T480 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 18702 T415 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 18706 T415 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57499_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57499"}
[junit4:junit4]   2> 18745 T420 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18745 T479 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18745 T433 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18745 T463 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18745 T448 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18745 T414 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19200 T480 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 19213 T480 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 19217 T480 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 19242 T480 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 19248 T480 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 19253 T480 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 19255 T480 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 19257 T480 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359964192127/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty4/
[junit4:junit4]   2> 19257 T480 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c1a7261
[junit4:junit4]   2> 19258 T480 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 19259 T480 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty4 forceNew:false
[junit4:junit4]   2> 19259 T480 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty4
[junit4:junit4]   2> 19259 T480 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty4/index/
[junit4:junit4]   2> 19260 T480 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 19261 T480 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty4/index forceNew:false
[junit4:junit4]   2> 19273 T480 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@791e2d7d),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 19274 T480 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 19274 T480 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty4/index
[junit4:junit4]   2> 19275 T480 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 19276 T480 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 19276 T480 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 19277 T480 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 19277 T480 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 19277 T480 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 19278 T480 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 19278 T480 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 19279 T480 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 19279 T480 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 19279 T480 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 19280 T480 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 19280 T480 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 19281 T480 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 19281 T480 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 19281 T480 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 19281 T480 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 19282 T480 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 19282 T480 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 19283 T480 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 19283 T480 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 19283 T480 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 19284 T480 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 19284 T480 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 19285 T480 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 19285 T480 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 19285 T480 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 19296 T480 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 19300 T480 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 19302 T480 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 19305 T480 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty4
[junit4:junit4]   2> 19306 T480 oass.SolrIndexSearcher.<init> Opening Searcher@3fdd7b0f main
[junit4:junit4]   2> 19308 T480 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 19308 T480 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 19308 T480 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 19316 T480 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 19362 T481 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3fdd7b0f main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 19366 T480 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 19366 T480 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 20255 T415 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 20257 T415 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57503_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57503"}
[junit4:junit4]   2> 20257 T415 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 20257 T415 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 20296 T479 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20296 T463 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20296 T433 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20296 T448 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20296 T420 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20296 T414 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20370 T480 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 20370 T480 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:57503 collection:collection1 shard:shard2
[junit4:junit4]   2> 20376 T480 oasc.ZkController.register We are http://127.0.0.1:57503/collection1/ and leader is http://127.0.0.1:57496/collection1/
[junit4:junit4]   2> 20376 T480 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57503
[junit4:junit4]   2> 20377 T480 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 20377 T480 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C67 name=collection1 org.apache.solr.core.SolrCore@1e455cba url=http://127.0.0.1:57503/collection1 node=127.0.0.1:57503_ C67_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:57503_, base_url=http://127.0.0.1:57503}
[junit4:junit4]   2> 20377 T482 C67 P57503 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 20378 T482 C67 P57503 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 20378 T482 C67 P57503 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 20378 T482 C67 P57503 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 20378 T480 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 20381 T482 C67 P57503 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 20382 T392 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 20382 T392 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 20383 T392 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 20410 T392 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 20413 T392 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4]   2> 20415 T392 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C66_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:57499_, base_url=http://127.0.0.1:57499}
[junit4:junit4]   2> 20768 T466 C66 P57499 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:57493/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 20778 T466 C66 P57499 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 20797 T466 C66 P57499 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:57499 START replicas=[http://127.0.0.1:57493/collection1/] nUpdates=100
[junit4:junit4]   2> 20798 T466 C66 P57499 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 20798 T466 C66 P57499 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 20799 T466 C66 P57499 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 20799 T466 C66 P57499 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 20799 T466 C66 P57499 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 20800 T466 C66 P57499 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:57493/collection1/. core=collection1
[junit4:junit4]   2> 20800 T466 C66 P57499 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C68 name=collection1 org.apache.solr.core.SolrCore@b3fb8af url=http://127.0.0.1:57493/collection1 node=127.0.0.1:57493_ C68_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:57493_, base_url=http://127.0.0.1:57493, leader=true}
[junit4:junit4]   2> 20816 T424 C68 P57493 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 20838 T425 C68 P57493 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 20840 T425 C68 P57493 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty1
[junit4:junit4]   2> 20844 T425 C68 P57493 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b20f25),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 20845 T425 C68 P57493 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 20852 T425 C68 P57493 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b20f25),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b20f25),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 20853 T425 C68 P57493 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 20855 T425 C68 P57493 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty1
[junit4:junit4]   2> 20855 T425 C68 P57493 oass.SolrIndexSearcher.<init> Opening Searcher@4d2457b1 realtime
[junit4:junit4]   2> 20856 T425 C68 P57493 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 20857 T425 C68 P57493 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 37
[junit4:junit4]   2> 20859 T466 C66 P57499 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 20859 T466 C66 P57499 oascsi.HttpClientUtil.createClient Creating new http client, config:connTimeout=5000&socketTimeout=20000&allowCompression=false&maxConnections=10000&maxConnectionsPerHost=10000
[junit4:junit4]   2> 20892 T466 C66 P57499 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 20897 T426 C68 P57493 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 20898 T426 C68 P57493 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 20899 T466 C66 P57499 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 20899 T466 C66 P57499 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 20900 T466 C66 P57499 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 20903 T426 C68 P57493 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty1
[junit4:junit4]   2> 20904 T426 C68 P57493 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty1/index
[junit4:junit4]   2> 20904 T426 C68 P57493 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=2 
[junit4:junit4]   2> 20905 T466 C66 P57499 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 20907 T466 C66 P57499 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty3/index.20130204074955295 forceNew:false
[junit4:junit4]   2> 20907 T466 C66 P57499 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty3
[junit4:junit4]   2> 20908 T466 C66 P57499 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty3/index.20130204074955295 lockFactory=org.apache.lucene.store.NativeFSLockFactory@e872d2f) fullCopy=true
[junit4:junit4]   2> 20925 T426 C68 P57493 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=7 
[junit4:junit4]   2> 20947 T466 C66 P57499 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty3/index
[junit4:junit4]   2> 20947 T466 C66 P57499 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 20948 T466 C66 P57499 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130204074955295
[junit4:junit4]   2> 20950 T466 C66 P57499 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty3
[junit4:junit4]   2> 20951 T466 C66 P57499 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty3
[junit4:junit4]   2> 20951 T466 C66 P57499 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty3/index/ new=./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty3/index.20130204074955295
[junit4:junit4]   2> 20955 T466 C66 P57499 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty3/index.20130204074955295 lockFactory=org.apache.lucene.store.NativeFSLockFactory@e872d2f),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 20956 T466 C66 P57499 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 20958 T466 C66 P57499 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty3
[junit4:junit4]   2> 20962 T466 C66 P57499 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 20963 T466 C66 P57499 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 20963 T466 C66 P57499 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 20964 T466 C66 P57499 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty3/index.20130204074955295
[junit4:junit4]   2> 20965 T466 C66 P57499 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty3
[junit4:junit4]   2> 20969 T466 C66 P57499 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty3/index.20130204074955295 lockFactory=org.apache.lucene.store.NativeFSLockFactory@e872d2f),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 20969 T466 C66 P57499 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 20969 T466 C66 P57499 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 20970 T466 C66 P57499 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty3
[junit4:junit4]   2> 20970 T466 C66 P57499 oass.SolrIndexSearcher.<init> Opening Searcher@7f1654e7 main
[junit4:junit4]   2> 20972 T465 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7f1654e7 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 20973 T465 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty3/index
[junit4:junit4]   2> 20973 T466 C66 P57499 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 20976 T466 C66 P57499 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty3/index.20130204074955295 lockFactory=org.apache.lucene.store.NativeFSLockFactory@e872d2f),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty3/index.20130204074955295 lockFactory=org.apache.lucene.store.NativeFSLockFactory@e872d2f),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 20977 T466 C66 P57499 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 20978 T466 C66 P57499 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty3
[junit4:junit4]   2> 20978 T466 C66 P57499 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 20979 T466 C66 P57499 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty3/index.20130204074955295
[junit4:junit4]   2> 20979 T466 C66 P57499 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty3/index
[junit4:junit4]   2> 20979 T466 C66 P57499 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 20979 T466 C66 P57499 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 20980 T466 C66 P57499 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 20980 T466 C66 P57499 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 20982 T466 C66 P57499 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 21418 T392 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 21809 T415 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 21811 T415 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57503_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57503"}
[junit4:junit4]   2> 21824 T415 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:57499_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:57499"}
[junit4:junit4]   2> 21837 T479 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 21837 T420 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 21837 T433 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 21837 T448 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 21837 T463 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 21837 T414 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 22421 T392 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 23425 T392 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C67_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:57503_, base_url=http://127.0.0.1:57503}
[junit4:junit4]   2> 24414 T482 C67 P57503 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:57496/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 24414 T482 C67 P57503 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:57503 START replicas=[http://127.0.0.1:57496/collection1/] nUpdates=100
[junit4:junit4]   2> 24415 T482 C67 P57503 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 24415 T482 C67 P57503 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 24415 T482 C67 P57503 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 24415 T482 C67 P57503 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 24416 T482 C67 P57503 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 24416 T482 C67 P57503 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:57496/collection1/. core=collection1
[junit4:junit4]   2> 24417 T482 C67 P57503 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 24428 T392 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> ASYNC  NEW_CORE C69 name=collection1 org.apache.solr.core.SolrCore@4f903ce5 url=http://127.0.0.1:57496/collection1 node=127.0.0.1:57496_ C69_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:57496_, base_url=http://127.0.0.1:57496, leader=true}
[junit4:junit4]   2> 24433 T439 C69 P57496 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1 
[junit4:junit4]   2> 24439 T440 C69 P57496 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 24440 T440 C69 P57496 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty2
[junit4:junit4]   2> 24444 T440 C69 P57496 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@64eb9eaf),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 24445 T440 C69 P57496 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 24449 T440 C69 P57496 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@64eb9eaf),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@64eb9eaf),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 24449 T440 C69 P57496 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 24450 T440 C69 P57496 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359964174397/jetty2
[junit4:junit4]   2> 24451 T440 C69 P57496 oass.SolrIndexSearcher.<init> Opening Searcher@1f14c6d0 realtime
[junit4:junit4]   2> 24451 T440 C69 P57496 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 24452 T440 C69 P57496 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 13
[junit4:junit4]   2> 24453 T482 C67 P57503 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 24454 T482 C67 P57503 oash.SnapPuller.<init>  No value set f

[...truncated too long message...]

ad.run(Thread.java:722)
[junit4:junit4]   2> 	  17) Thread[id=421, name=HashSessionScavenger-10, state=TIMED_WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at java.lang.Object.wait(Native Method)
[junit4:junit4]   2> 	        at java.util.TimerThread.mainLoop(Timer.java:552)
[junit4:junit4]   2> 	        at java.util.TimerThread.run(Timer.java:505)
[junit4:junit4]   2> 	  18) Thread[id=496, name=TEST-BasicDistributedZkTest.testDistribSearch-seed#[D57BB880CB873303]-SendThread(localhost:57486), state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:82)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
[junit4:junit4]   2> 	  19) Thread[id=478, name=TEST-BasicDistributedZkTest.testDistribSearch-seed#[D57BB880CB873303]-SendThread(localhost:57486), state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:82)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
[junit4:junit4]   2> 	  20) Thread[id=422, name=qtp934902858-422 Acceptor0 SocketConnector@127.0.0.1:57493, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at java.net.PlainSocketImpl.socketAccept(Native Method)
[junit4:junit4]   2> 	        at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
[junit4:junit4]   2> 	        at java.net.ServerSocket.implAccept(ServerSocket.java:522)
[junit4:junit4]   2> 	        at java.net.ServerSocket.accept(ServerSocket.java:490)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.bio.SocketConnector.accept(SocketConnector.java:117)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  21) Thread[id=447, name=TEST-BasicDistributedZkTest.testDistribSearch-seed#[D57BB880CB873303]-SendThread(localhost:57486), state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at java.nio.channels.spi.AbstractSelector.begin(AbstractSelector.java:216)
[junit4:junit4]   2> 	        at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:102)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
[junit4:junit4]   2> 	  22) Thread[id=538, name=searcherExecutor-441-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  23) Thread[id=462, name=TEST-BasicDistributedZkTest.testDistribSearch-seed#[D57BB880CB873303]-SendThread(localhost:57486), state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:159)
[junit4:junit4]   2> 	        at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:103)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
[junit4:junit4]   2> 	  24) Thread[id=541, name=searcherExecutor-449-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> NOTE: test params are: codec=Lucene42: {n_f1=PostingsFormat(name=Direct), foo_b=Lucene41(blocksize=128), foo_d=PostingsFormat(name=MockSep), foo_f=Lucene41(blocksize=128), n_tl1=PostingsFormat(name=Direct), n_d1=MockFixedIntBlock(blockSize=1134), rnd_b=PostingsFormat(name=MockSep), intDefault=PostingsFormat(name=Direct), n_td1=PostingsFormat(name=Direct), timestamp=MockFixedIntBlock(blockSize=1134), id=PostingsFormat(name=MockSep), range_facet_sl=PostingsFormat(name=MockSep), range_facet_si=MockFixedIntBlock(blockSize=1134), oddField_s=PostingsFormat(name=MockSep), foo_i=PostingsFormat(name=Direct), multiDefault=MockFixedIntBlock(blockSize=1134), n_tf1=MockFixedIntBlock(blockSize=1134), n_dt1=PostingsFormat(name=Direct), n_ti1=PostingsFormat(name=MockSep), range_facet_l=MockFixedIntBlock(blockSize=1134), text=Lucene41(blocksize=128), _version_=PostingsFormat(name=Direct), val_i=PostingsFormat(name=MockSep), SubjectTerms_mfacet=PostingsFormat(name=Direct), a_t=MockFixedIntBlock(blockSize=1134), n_tdt1=PostingsFormat(name=Direct), other_tl1=PostingsFormat(name=Direct), n_l1=MockFixedIntBlock(blockSize=1134), a_si=PostingsFormat(name=Direct)}, sim=DefaultSimilarity, locale=ar_SA, timezone=Atlantic/Faeroe
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=26,free=99856064,total=260046848
[junit4:junit4]   2> NOTE: All tests run in this JVM: [CSVRequestHandlerTest, TestBinaryResponseWriter, TestIndexingPerformance, TestFastOutputStream, TestSearchPerf, ResourceLoaderTest, JSONWriterTest, FileBasedSpellCheckerTest, TestBadConfig, TestExtendedDismaxParser, SuggesterFSTTest, DateMathParserTest, TestReload, SolrCoreCheckLockOnStartupTest, DefaultValueUpdateProcessorTest, TestCoreContainer, JsonLoaderTest, NumericFieldsTest, NotRequiredUniqueKeyTest, ExternalFileFieldSortTest, TestRandomFaceting, DistributedTermsComponentTest, SuggesterWFSTTest, TestSolrQueryParser, BinaryUpdateRequestHandlerTest, BasicDistributedZkTest]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=BasicDistributedZkTest -Dtests.seed=D57BB880CB873303 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ar_SA -Dtests.timezone=Atlantic/Faeroe -Dtests.file.encoding=ISO-8859-1
[junit4:junit4] ERROR   0.00s J0 | BasicDistributedZkTest (suite) <<<
[junit4:junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([D57BB880CB873303]:0)
[junit4:junit4] Completed on J0 in 7223.50s, 1 test, 2 errors <<< FAILURES!

[...truncated 116 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:346: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/common-build.xml:428: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:1176: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:841: There were test failures: 249 suites, 1038 tests, 1 suite-level error, 1 error, 146 ignored (4 assumptions)

Total time: 146 minutes 17 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



Re: [JENKINS] Lucene-Solr-Tests-trunk-java7 - Build # 3705 - Still Failing

Posted by Dawid Weiss <da...@cs.put.poznan.pl>.
Very weird.

On Mon, Feb 4, 2013 at 12:46 AM, Mark Miller <ma...@gmail.com> wrote:
> It looks like this thing hung doing a security check….
>
> [junit4:junit4]   2>      21) Thread[id=499, name=TEST-BasicDistributedZkTest.testDistribSearch-seed#[437C1ACFB7351B0A], state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
> [junit4:junit4]   2>            at java.security.AccessController.getStackAccessControlContext(Native Method)
> [junit4:junit4]   2>            at java.security.AccessController.checkPermission(AccessController.java:534)
> [junit4:junit4]   2>            at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
> [junit4:junit4]   2>            at java.util.concurrent.ThreadPoolExecutor.checkShutdownAccess(ThreadPoolExecutor.java:711)
> [junit4:junit4]   2>            at java.util.concurrent.ThreadPoolExecutor.shutdownNow(ThreadPoolExecutor.java:1387)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Re: [JENKINS] Lucene-Solr-Tests-trunk-java7 - Build # 3705 - Still Failing

Posted by Mark Miller <ma...@gmail.com>.
It looks like this thing hung doing a security check….

[junit4:junit4]   2> 	  21) Thread[id=499, name=TEST-BasicDistributedZkTest.testDistribSearch-seed#[437C1ACFB7351B0A], state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at java.security.AccessController.getStackAccessControlContext(Native Method)
[junit4:junit4]   2> 	        at java.security.AccessController.checkPermission(AccessController.java:534)
[junit4:junit4]   2> 	        at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.checkShutdownAccess(ThreadPoolExecutor.java:711)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.shutdownNow(ThreadPoolExecutor.java:1387)
[junit4:junit4]   2> 	        at org.apache.solr.common.util.ExecutorUtil.shutdownNowAndAwaitTermination(ExecutorUtil.java:43)
[junit4:junit4]   2> 	        at org.apache.solr.update.UpdateShardHandler.close(UpdateShardHandler.java:63)
[junit4:junit4]   2> 	        at org.apache.solr.cloud.ZkController.close(ZkController.java:359)
[junit4:junit4]   2> 	        at org.apache.solr.core.CoreContainer.shutdown(CoreContainer.java:777)
[junit4:junit4]   2> 	        at org.apache.solr.servlet.SolrDispatchFilter.destroy(SolrDispatchFilter.java:122)
[junit4:junit4]   2> 	        at org.apache.solr.cloud.ChaosMonkey.stopJettySolrRunner(ChaosMonkey.java:202)
[junit4:junit4]   2> 	        at org.apache.solr.cloud.ChaosMonkey.stop(ChaosMonkey.java:509)
[junit4:junit4]   2> 	        at org.apache.solr.cloud.AbstractFullDistribZkTestBase.destroyServers(AbstractFullDistribZkTestBase.java:1329)
[junit4:junit4]   2> 	        at org.apache.solr.BaseDistributedSearchTestCase.tearDown(BaseDistributedSearchTestCase.java:280)
[junit4:junit4]   2> 	        at org.apache.solr.cloud.AbstractDistribZkTestBase.tearDown(AbstractDistribZkTestBase.java:219)
[junit4:junit4]   2> 	        at org.apache.solr.cloud.AbstractFullDistribZkTestBase.tearDown(AbstractFullDistribZkTestBase.java:1310)
[junit4:junit4]   2> 	        at org.apache.solr.cloud.BasicDistributedZkTest.tearDown(BasicDistributedZkTest.java:1370)

On Feb 3, 2013, at 6:15 PM, Apache Jenkins Server <je...@builds.apache.org> wrote:

> Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-java7/3705/
> 
> 2 tests failed.
> FAILED:  org.apache.solr.cloud.BasicDistributedZkTest.testDistribSearch
> 
> Error Message:
> Test abandoned because suite timeout was reached.
> 
> Stack Trace:
> java.lang.Exception: Test abandoned because suite timeout was reached.
> 	at __randomizedtesting.SeedInfo.seed([437C1ACFB7351B0A]:0)
> 
> 
> FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest
> 
> Error Message:
> Suite timeout exceeded (>= 7200000 msec).
> 
> Stack Trace:
> java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
> 	at __randomizedtesting.SeedInfo.seed([437C1ACFB7351B0A]:0)
> 
> 
> 
> 
> Build Log:
> [...truncated 9446 lines...]
> [junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
> [junit4:junit4]   2> 1 T499 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /lt_/
> [junit4:junit4]   2> 6 T499 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
> [junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZkTest-1359926089911
> [junit4:junit4]   2> 7 T499 oasc.ZkTestServer.run STARTING ZK TEST SERVER
> [junit4:junit4]   2> 8 T500 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
> [junit4:junit4]   2> 10 T500 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
> [junit4:junit4]   2> 10 T500 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
> [junit4:junit4]   2> 10 T500 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
> [junit4:junit4]   2> 11 T500 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
> [junit4:junit4]   2> 11 T500 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/zookeeper/server1/data/version-2/snapshot.0
> [junit4:junit4]   2> 109 T499 oasc.ZkTestServer.run start zk server on port:43537
> [junit4:junit4]   2> 109 T499 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:43537 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@40f81bb4
> [junit4:junit4]   2> 110 T499 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4]   2> 111 T505 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:43537. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4]   2> 112 T505 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:43537, initiating session
> [junit4:junit4]   2> 112 T501 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:43538
> [junit4:junit4]   2> 112 T501 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:43538
> [junit4:junit4]   2> 113 T503 oazsp.FileTxnLog.append Creating new log file: log.1
> [junit4:junit4]   2> 117 T503 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca1e958bc0000 with negotiated timeout 10000 for client /140.211.11.196:43538
> [junit4:junit4]   2> 117 T505 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:43537, sessionid = 0x13ca1e958bc0000, negotiated timeout = 10000
> [junit4:junit4]   2> 117 T506 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@40f81bb4 name:ZooKeeperConnection Watcher:127.0.0.1:43537 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4]   2> 117 T499 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4]   2> 118 T499 oascc.SolrZkClient.makePath makePath: /solr
> [junit4:junit4]   2> 121 T504 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca1e958bc0000
> [junit4:junit4]   2> 133 T506 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4]   2> 133 T499 oaz.ZooKeeper.close Session: 0x13ca1e958bc0000 closed
> [junit4:junit4]   2> 134 T501 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:43538 which had sessionid 0x13ca1e958bc0000
> [junit4:junit4]   2> 134 T499 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:43537/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@70df8d56
> [junit4:junit4]   2> 135 T499 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4]   2> 135 T507 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:43537. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4]   2> 136 T507 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:43537, initiating session
> [junit4:junit4]   2> 136 T501 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:43539
> [junit4:junit4]   2> 136 T501 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:43539
> [junit4:junit4]   2> 160 T503 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca1e958bc0001 with negotiated timeout 10000 for client /140.211.11.196:43539
> [junit4:junit4]   2> 160 T507 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:43537, sessionid = 0x13ca1e958bc0001, negotiated timeout = 10000
> [junit4:junit4]   2> 160 T508 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@70df8d56 name:ZooKeeperConnection Watcher:127.0.0.1:43537/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4]   2> 160 T499 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4]   2> 161 T499 oascc.SolrZkClient.makePath makePath: /collections/collection1
> [junit4:junit4]   2> 166 T499 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
> [junit4:junit4]   2> 175 T499 oascc.SolrZkClient.makePath makePath: /collections/control_collection
> [junit4:junit4]   2> 178 T499 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
> [junit4:junit4]   2> 182 T499 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
> [junit4:junit4]   2> 183 T499 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
> [junit4:junit4]   2> 196 T499 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
> [junit4:junit4]   2> 197 T499 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
> [junit4:junit4]   2> 302 T499 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
> [junit4:junit4]   2> 302 T499 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
> [junit4:junit4]   2> 306 T499 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
> [junit4:junit4]   2> 307 T499 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
> [junit4:junit4]   2> 310 T499 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
> [junit4:junit4]   2> 311 T499 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
> [junit4:junit4]   2> 315 T499 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
> [junit4:junit4]   2> 315 T499 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
> [junit4:junit4]   2> 319 T499 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
> [junit4:junit4]   2> 320 T499 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
> [junit4:junit4]   2> 323 T499 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
> [junit4:junit4]   2> 324 T499 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
> [junit4:junit4]   2> 327 T499 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
> [junit4:junit4]   2> 328 T499 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
> [junit4:junit4]   2> 331 T504 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca1e958bc0001
> [junit4:junit4]   2> 338 T508 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4]   2> 338 T501 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:43539 which had sessionid 0x13ca1e958bc0001
> [junit4:junit4]   2> 338 T499 oaz.ZooKeeper.close Session: 0x13ca1e958bc0001 closed
> [junit4:junit4]   2> 486 T499 oejs.Server.doStart jetty-8.1.8.v20121106
> [junit4:junit4]   2> 490 T499 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:25591
> [junit4:junit4]   2> 490 T499 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4:junit4]   2> 491 T499 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4:junit4]   2> 491 T499 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359926090243
> [junit4:junit4]   2> 492 T499 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359926090243/solr.xml
> [junit4:junit4]   2> 492 T499 oasc.CoreContainer.<init> New CoreContainer 2073771471
> [junit4:junit4]   2> 492 T499 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359926090243/'
> [junit4:junit4]   2> 493 T499 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359926090243/'
> [junit4:junit4]   2> 562 T499 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
> [junit4:junit4]   2> 562 T499 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4:junit4]   2> 563 T499 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4:junit4]   2> 563 T499 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4:junit4]   2> 564 T499 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4:junit4]   2> 564 T499 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4:junit4]   2> 565 T499 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4:junit4]   2> 565 T499 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4:junit4]   2> 566 T499 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4:junit4]   2> 566 T499 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
> [junit4:junit4]   2> 590 T499 oasc.CoreContainer.load Registering Log Listener
> [junit4:junit4]   2> 610 T499 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:43537/solr
> [junit4:junit4]   2> 610 T499 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4:junit4]   2> 611 T499 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:43537 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@3daa432
> [junit4:junit4]   2> 612 T499 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4]   2> 612 T518 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:43537. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4]   2> 613 T518 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:43537, initiating session
> [junit4:junit4]   2> 614 T501 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:55204
> [junit4:junit4]   2> 614 T501 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:55204
> [junit4:junit4]   2> 619 T503 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca1e958bc0002 with negotiated timeout 20000 for client /140.211.11.196:55204
> [junit4:junit4]   2> 619 T518 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:43537, sessionid = 0x13ca1e958bc0002, negotiated timeout = 20000
> [junit4:junit4]   2> 619 T519 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3daa432 name:ZooKeeperConnection Watcher:127.0.0.1:43537 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4]   2> 619 T499 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4]   2> 621 T504 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca1e958bc0002
> [junit4:junit4]   2> 622 T519 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4]   2> 622 T501 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:55204 which had sessionid 0x13ca1e958bc0002
> [junit4:junit4]   2> 622 T499 oaz.ZooKeeper.close Session: 0x13ca1e958bc0002 closed
> [junit4:junit4]   2> 623 T499 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
> [junit4:junit4]   2> 637 T499 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:43537/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@7eb5d65e
> [junit4:junit4]   2> 638 T499 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4]   2> 638 T520 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:43537. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4]   2> 639 T520 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:43537, initiating session
> [junit4:junit4]   2> 639 T501 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:45054
> [junit4:junit4]   2> 639 T501 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:45054
> [junit4:junit4]   2> 641 T503 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca1e958bc0003 with negotiated timeout 20000 for client /140.211.11.196:45054
> [junit4:junit4]   2> 641 T520 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:43537, sessionid = 0x13ca1e958bc0003, negotiated timeout = 20000
> [junit4:junit4]   2> 641 T521 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7eb5d65e name:ZooKeeperConnection Watcher:127.0.0.1:43537/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4]   2> 641 T499 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4]   2> 644 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 652 T499 oascc.SolrZkClient.makePath makePath: /live_nodes
> [junit4:junit4]   2> 654 T499 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:25591_lt_
> [junit4:junit4]   2> 655 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0003 type:delete cxid:0x7 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:25591_lt_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:25591_lt_
> [junit4:junit4]   2> 657 T499 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:25591_lt_
> [junit4:junit4]   2> 667 T499 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
> [junit4:junit4]   2> 680 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0003 type:delete cxid:0x15 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
> [junit4:junit4]   2> 681 T499 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
> [junit4:junit4]   2> 685 T499 oasc.Overseer.start Overseer (id=89124116228734979-127.0.0.1:25591_lt_-n_0000000000) starting
> [junit4:junit4]   2> 685 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0003 type:create cxid:0x1a zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 691 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0003 type:create cxid:0x1b zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 693 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0003 type:create cxid:0x1c zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 699 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0003 type:create cxid:0x1d zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 701 T523 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
> [junit4:junit4]   2> 702 T499 oascc.SolrZkClient.makePath makePath: /clusterstate.json
> [junit4:junit4]   2> 705 T499 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
> [junit4:junit4]   2> 708 T522 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
> [junit4:junit4]   2> 720 T524 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359926090243/collection1
> [junit4:junit4]   2> 720 T524 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
> [junit4:junit4]   2> 721 T524 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4:junit4]   2> 722 T524 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
> [junit4:junit4]   2> 724 T524 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359926090243/collection1/'
> [junit4:junit4]   2> 725 T524 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359926090243/collection1/lib/README' to classloader
> [junit4:junit4]   2> 726 T524 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359926090243/collection1/lib/classes/' to classloader
> [junit4:junit4]   2> 782 T524 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4:junit4]   2> 854 T524 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4:junit4]   2> 955 T524 oass.IndexSchema.readSchema Reading Solr Schema
> [junit4:junit4]   2> 961 T524 oass.IndexSchema.readSchema Schema name=test
> [junit4:junit4]   2> 1569 T524 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4:junit4]   2> 1577 T524 oass.IndexSchema.readSchema default search field in schema is text
> [junit4:junit4]   2> 1580 T524 oass.IndexSchema.readSchema unique key field: id
> [junit4:junit4]   2> 1596 T524 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4]   2> 1600 T524 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4]   2> 1603 T524 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4]   2> 1605 T524 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4]   2> 1606 T524 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359926090243/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/control/data/
> [junit4:junit4]   2> 1606 T524 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@36298619
> [junit4:junit4]   2> 1607 T524 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
> [junit4:junit4]   2> 1630 T524 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/control/data forceNew:false
> [junit4:junit4]   2> 1630 T524 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/control/data
> [junit4:junit4]   2> 1630 T524 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/control/data/index/
> [junit4:junit4]   2> 1631 T524 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/control/data/index' doesn't exist. Creating new index...
> [junit4:junit4]   2> 1632 T524 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/control/data/index forceNew:false
> [junit4:junit4]   2> 1638 T524 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@708ce5ef; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4]   2> 1638 T524 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
> [junit4:junit4]   2> 1639 T524 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/control/data/index
> [junit4:junit4]   2> 1639 T524 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
> [junit4:junit4]   2> 1641 T524 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4:junit4]   2> 1641 T524 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
> [junit4:junit4]   2> 1641 T524 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4:junit4]   2> 1641 T524 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
> [junit4:junit4]   2> 1642 T524 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4:junit4]   2> 1642 T524 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4:junit4]   2> 1642 T524 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4:junit4]   2> 1642 T524 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4:junit4]   2> 1643 T524 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4:junit4]   2> 1643 T524 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
> [junit4:junit4]   2> 1644 T524 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
> [junit4:junit4]   2> 1644 T524 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4:junit4]   2> 1644 T524 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
> [junit4:junit4]   2> 1644 T524 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
> [junit4:junit4]   2> 1645 T524 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
> [junit4:junit4]   2> 1645 T524 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4:junit4]   2> 1645 T524 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 1646 T524 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 1646 T524 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 1646 T524 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 1647 T524 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 1647 T524 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 1647 T524 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 1648 T524 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 1648 T524 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
> [junit4:junit4]   2> 1648 T524 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
> [junit4:junit4]   2> 1657 T524 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4:junit4]   2> 1660 T524 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
> [junit4:junit4]   2> 1661 T524 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
> [junit4:junit4]   2> 1663 T524 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/control/data
> [junit4:junit4]   2> 1664 T524 oass.SolrIndexSearcher.<init> Opening Searcher@4b06504d main
> [junit4:junit4]   2> 1665 T524 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4:junit4]   2> 1665 T524 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4:junit4]   2> 1665 T524 oashc.SpellCheckComponent.inform Initializing spell checkers
> [junit4:junit4]   2> 1682 T524 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
> [junit4:junit4]   2> 1728 T525 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4b06504d main{StandardDirectoryReader(segments_1:1)}
> [junit4:junit4]   2> 1732 T524 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4]   2> 1732 T524 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4]   2> 1732 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0003 type:create cxid:0x50 zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
> [junit4:junit4]   2> 2212 T522 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 2213 T522 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
> [junit4:junit4]   2> 	  "operation":"state",
> [junit4:junit4]   2> 	  "numShards":"1",
> [junit4:junit4]   2> 	  "shard":null,
> [junit4:junit4]   2> 	  "roles":null,
> [junit4:junit4]   2> 	  "state":"down",
> [junit4:junit4]   2> 	  "core":"collection1",
> [junit4:junit4]   2> 	  "collection":"control_collection",
> [junit4:junit4]   2> 	  "node_name":"127.0.0.1:25591_lt_",
> [junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:25591/lt_"}
> [junit4:junit4]   2> 2213 T522 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
> [junit4:junit4]   2> 2226 T522 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
> [junit4:junit4]   2> 2226 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0003 type:create cxid:0x58 zxid:0x2c txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
> [junit4:junit4]   2> 2245 T521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4:junit4]   2> 2741 T524 oasc.CoreContainer.registerCore registering core: collection1
> [junit4:junit4]   2> 2741 T524 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:25591/lt_ collection:control_collection shard:shard1
> [junit4:junit4]   2> 2742 T524 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
> [junit4:junit4]   2> 2759 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0003 type:delete cxid:0x72 zxid:0x36 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
> [junit4:junit4]   2> 2760 T524 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
> [junit4:junit4]   2> 2760 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0003 type:create cxid:0x73 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 2771 T524 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
> [junit4:junit4]   2> 2771 T524 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
> [junit4:junit4]   2> 2771 T524 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:25591/lt_/collection1/
> [junit4:junit4]   2> 2772 T524 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
> [junit4:junit4]   2> 2772 T524 oasc.SyncStrategy.syncToMe http://127.0.0.1:25591/lt_/collection1/ has no replicas
> [junit4:junit4]   2> 2772 T524 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:25591/lt_/collection1/
> [junit4:junit4]   2> 2772 T524 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
> [junit4:junit4]   2> 2777 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0003 type:create cxid:0x7d zxid:0x3b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 3751 T522 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 3802 T521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4:junit4]   2> 3851 T524 oasc.ZkController.register We are http://127.0.0.1:25591/lt_/collection1/ and leader is http://127.0.0.1:25591/lt_/collection1/
> [junit4:junit4]   2> 3851 T524 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:25591/lt_
> [junit4:junit4]   2> 3851 T524 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
> [junit4:junit4]   2> 3851 T524 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4:junit4]   2> 3852 T524 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4]   2> 3854 T524 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 3855 T499 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
> [junit4:junit4]   2> 3856 T499 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4:junit4]   2> 3857 T499 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4]   2> 3873 T499 oascsi.HttpClientUtil.createClient Creating new http client, config:
> [junit4:junit4]   2> 3890 T499 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:43537/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@417dbcb6
> [junit4:junit4]   2> 3891 T499 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4]   2> 3892 T526 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:43537. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4]   2> 3892 T526 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:43537, initiating session
> [junit4:junit4]   2> 3893 T501 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:40197
> [junit4:junit4]   2> 3893 T501 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:40197
> [junit4:junit4]   2> 3895 T503 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca1e958bc0004 with negotiated timeout 10000 for client /140.211.11.196:40197
> [junit4:junit4]   2> 3895 T526 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:43537, sessionid = 0x13ca1e958bc0004, negotiated timeout = 10000
> [junit4:junit4]   2> 3895 T527 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@417dbcb6 name:ZooKeeperConnection Watcher:127.0.0.1:43537/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4]   2> 3895 T499 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4]   2> 3897 T499 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
> [junit4:junit4]   2> 3901 T499 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
> [junit4:junit4]   2> 4069 T499 oejs.Server.doStart jetty-8.1.8.v20121106
> [junit4:junit4]   2> 4073 T499 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:59558
> [junit4:junit4]   2> 4073 T499 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4:junit4]   2> 4074 T499 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4:junit4]   2> 4074 T499 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359926093806
> [junit4:junit4]   2> 4075 T499 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359926093806/solr.xml
> [junit4:junit4]   2> 4075 T499 oasc.CoreContainer.<init> New CoreContainer 2041002222
> [junit4:junit4]   2> 4076 T499 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359926093806/'
> [junit4:junit4]   2> 4076 T499 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359926093806/'
> [junit4:junit4]   2> 4131 T499 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
> [junit4:junit4]   2> 4132 T499 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4:junit4]   2> 4132 T499 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4:junit4]   2> 4132 T499 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4:junit4]   2> 4133 T499 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4:junit4]   2> 4133 T499 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4:junit4]   2> 4134 T499 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4:junit4]   2> 4134 T499 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4:junit4]   2> 4135 T499 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4:junit4]   2> 4135 T499 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
> [junit4:junit4]   2> 4157 T499 oasc.CoreContainer.load Registering Log Listener
> [junit4:junit4]   2> 4176 T499 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:43537/solr
> [junit4:junit4]   2> 4176 T499 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4:junit4]   2> 4177 T499 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:43537 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@14c6f4dd
> [junit4:junit4]   2> 4178 T499 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4]   2> 4178 T537 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:43537. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4]   2> 4179 T537 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:43537, initiating session
> [junit4:junit4]   2> 4180 T501 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:35606
> [junit4:junit4]   2> 4180 T501 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:35606
> [junit4:junit4]   2> 4182 T503 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca1e958bc0005 with negotiated timeout 20000 for client /140.211.11.196:35606
> [junit4:junit4]   2> 4182 T537 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:43537, sessionid = 0x13ca1e958bc0005, negotiated timeout = 20000
> [junit4:junit4]   2> 4182 T538 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14c6f4dd name:ZooKeeperConnection Watcher:127.0.0.1:43537 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4]   2> 4183 T499 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4]   2> 4184 T504 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca1e958bc0005
> [junit4:junit4]   2> 4193 T538 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4]   2> 4193 T501 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:35606 which had sessionid 0x13ca1e958bc0005
> [junit4:junit4]   2> 4193 T499 oaz.ZooKeeper.close Session: 0x13ca1e958bc0005 closed
> [junit4:junit4]   2> 4194 T499 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
> [junit4:junit4]   2> 4205 T499 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:43537/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@4d111d50
> [junit4:junit4]   2> 4206 T499 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4]   2> 4207 T539 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:43537. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4]   2> 4207 T539 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:43537, initiating session
> [junit4:junit4]   2> 4208 T501 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:25987
> [junit4:junit4]   2> 4208 T501 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:25987
> [junit4:junit4]   2> 4209 T503 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca1e958bc0006 with negotiated timeout 20000 for client /140.211.11.196:25987
> [junit4:junit4]   2> 4209 T539 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:43537, sessionid = 0x13ca1e958bc0006, negotiated timeout = 20000
> [junit4:junit4]   2> 4210 T540 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4d111d50 name:ZooKeeperConnection Watcher:127.0.0.1:43537/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4]   2> 4210 T499 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4]   2> 4211 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0006 type:create cxid:0x1 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 4213 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0006 type:create cxid:0x2 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 4214 T499 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59558_lt_
> [junit4:junit4]   2> 4215 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0006 type:delete cxid:0x4 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:59558_lt_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:59558_lt_
> [junit4:junit4]   2> 4222 T499 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:59558_lt_
> [junit4:junit4]   2> 4225 T521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
> [junit4:junit4]   2> 4226 T527 oascc.ZkStateReader$3.process Updating live nodes... (2)
> [junit4:junit4]   2> 4228 T521 oascc.ZkStateReader$3.process Updating live nodes... (2)
> [junit4:junit4]   2> 4230 T499 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
> [junit4:junit4]   2> 4238 T541 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359926093806/collection1
> [junit4:junit4]   2> 4238 T541 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
> [junit4:junit4]   2> 4239 T541 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4:junit4]   2> 4239 T541 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
> [junit4:junit4]   2> 4241 T541 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359926093806/collection1/'
> [junit4:junit4]   2> 4242 T541 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359926093806/collection1/lib/README' to classloader
> [junit4:junit4]   2> 4243 T541 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359926093806/collection1/lib/classes/' to classloader
> [junit4:junit4]   2> 4295 T541 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4:junit4]   2> 4366 T541 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4:junit4]   2> 4467 T541 oass.IndexSchema.readSchema Reading Solr Schema
> [junit4:junit4]   2> 4473 T541 oass.IndexSchema.readSchema Schema name=test
> [junit4:junit4]   2> 5077 T541 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4:junit4]   2> 5084 T541 oass.IndexSchema.readSchema default search field in schema is text
> [junit4:junit4]   2> 5087 T541 oass.IndexSchema.readSchema unique key field: id
> [junit4:junit4]   2> 5102 T541 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4]   2> 5106 T541 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4]   2> 5109 T541 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4]   2> 5111 T541 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4]   2> 5112 T541 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359926093806/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty1/
> [junit4:junit4]   2> 5112 T541 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@36298619
> [junit4:junit4]   2> 5112 T541 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
> [junit4:junit4]   2> 5113 T541 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty1 forceNew:false
> [junit4:junit4]   2> 5113 T541 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty1
> [junit4:junit4]   2> 5114 T541 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty1/index/
> [junit4:junit4]   2> 5114 T541 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty1/index' doesn't exist. Creating new index...
> [junit4:junit4]   2> 5115 T541 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty1/index forceNew:false
> [junit4:junit4]   2> 5119 T541 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7bf4515f; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4]   2> 5120 T541 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
> [junit4:junit4]   2> 5120 T541 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty1/index
> [junit4:junit4]   2> 5121 T541 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
> [junit4:junit4]   2> 5122 T541 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4:junit4]   2> 5122 T541 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
> [junit4:junit4]   2> 5123 T541 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4:junit4]   2> 5123 T541 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
> [junit4:junit4]   2> 5123 T541 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4:junit4]   2> 5124 T541 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4:junit4]   2> 5124 T541 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4:junit4]   2> 5124 T541 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4:junit4]   2> 5125 T541 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4:junit4]   2> 5125 T541 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
> [junit4:junit4]   2> 5126 T541 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
> [junit4:junit4]   2> 5126 T541 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4:junit4]   2> 5126 T541 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
> [junit4:junit4]   2> 5127 T541 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
> [junit4:junit4]   2> 5127 T541 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
> [junit4:junit4]   2> 5127 T541 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4:junit4]   2> 5128 T541 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 5128 T541 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 5129 T541 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 5129 T541 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 5129 T541 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 5130 T541 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 5130 T541 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 5131 T541 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 5131 T541 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
> [junit4:junit4]   2> 5132 T541 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
> [junit4:junit4]   2> 5140 T541 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4:junit4]   2> 5143 T541 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
> [junit4:junit4]   2> 5144 T541 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
> [junit4:junit4]   2> 5146 T541 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty1
> [junit4:junit4]   2> 5147 T541 oass.SolrIndexSearcher.<init> Opening Searcher@2aa0240c main
> [junit4:junit4]   2> 5148 T541 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4:junit4]   2> 5148 T541 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4:junit4]   2> 5148 T541 oashc.SpellCheckComponent.inform Initializing spell checkers
> [junit4:junit4]   2> 5153 T541 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
> [junit4:junit4]   2> 5196 T542 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2aa0240c main{StandardDirectoryReader(segments_1:1)}
> [junit4:junit4]   2> 5200 T541 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4]   2> 5200 T541 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4]   2> 5317 T522 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 5318 T522 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
> [junit4:junit4]   2> 	  "operation":"state",
> [junit4:junit4]   2> 	  "numShards":"1",
> [junit4:junit4]   2> 	  "shard":"shard1",
> [junit4:junit4]   2> 	  "roles":null,
> [junit4:junit4]   2> 	  "state":"active",
> [junit4:junit4]   2> 	  "core":"collection1",
> [junit4:junit4]   2> 	  "collection":"control_collection",
> [junit4:junit4]   2> 	  "node_name":"127.0.0.1:25591_lt_",
> [junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:25591/lt_"}
> [junit4:junit4]   2> 5322 T522 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4]   2> 	  "operation":"state",
> [junit4:junit4]   2> 	  "numShards":"2",
> [junit4:junit4]   2> 	  "shard":null,
> [junit4:junit4]   2> 	  "roles":null,
> [junit4:junit4]   2> 	  "state":"down",
> [junit4:junit4]   2> 	  "core":"collection1",
> [junit4:junit4]   2> 	  "collection":"collection1",
> [junit4:junit4]   2> 	  "node_name":"127.0.0.1:59558_lt_",
> [junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:59558/lt_"}
> [junit4:junit4]   2> 5322 T522 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
> [junit4:junit4]   2> 5323 T522 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
> [junit4:junit4]   2> 5334 T540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4]   2> 5334 T521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4]   2> 5334 T527 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4]   2> 6203 T541 oasc.CoreContainer.registerCore registering core: collection1
> [junit4:junit4]   2> 6203 T541 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:59558/lt_ collection:collection1 shard:shard1
> [junit4:junit4]   2> 6204 T541 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
> [junit4:junit4]   2> 6224 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0006 type:delete cxid:0x43 zxid:0x5a txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
> [junit4:junit4]   2> 6225 T541 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
> [junit4:junit4]   2> 6225 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0006 type:create cxid:0x44 zxid:0x5b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 6236 T541 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
> [junit4:junit4]   2> 6236 T541 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
> [junit4:junit4]   2> 6237 T541 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:59558/lt_/collection1/
> [junit4:junit4]   2> 6237 T541 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
> [junit4:junit4]   2> 6237 T541 oasc.SyncStrategy.syncToMe http://127.0.0.1:59558/lt_/collection1/ has no replicas
> [junit4:junit4]   2> 6237 T541 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:59558/lt_/collection1/
> [junit4:junit4]   2> 6238 T541 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
> [junit4:junit4]   2> 6243 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0006 type:create cxid:0x4e zxid:0x5f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 6848 T522 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 6892 T540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4]   2> 6892 T521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4]   2> 6892 T527 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4]   2> 6916 T541 oasc.ZkController.register We are http://127.0.0.1:59558/lt_/collection1/ and leader is http://127.0.0.1:59558/lt_/collection1/
> [junit4:junit4]   2> 6916 T541 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:59558/lt_
> [junit4:junit4]   2> 6916 T541 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
> [junit4:junit4]   2> 6916 T541 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4:junit4]   2> 6917 T541 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4]   2> 6919 T541 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 6920 T499 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
> [junit4:junit4]   2> 6920 T499 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4:junit4]   2> 6921 T499 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4]   2> 7104 T499 oejs.Server.doStart jetty-8.1.8.v20121106
> [junit4:junit4]   2> 7107 T499 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:39198
> [junit4:junit4]   2> 7108 T499 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4:junit4]   2> 7108 T499 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4:junit4]   2> 7109 T499 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359926096841
> [junit4:junit4]   2> 7109 T499 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359926096841/solr.xml
> [junit4:junit4]   2> 7110 T499 oasc.CoreContainer.<init> New CoreContainer 1233522068
> [junit4:junit4]   2> 7110 T499 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359926096841/'
> [junit4:junit4]   2> 7111 T499 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359926096841/'
> [junit4:junit4]   2> 7165 T499 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
> [junit4:junit4]   2> 7166 T499 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4:junit4]   2> 7166 T499 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4:junit4]   2> 7167 T499 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4:junit4]   2> 7167 T499 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4:junit4]   2> 7168 T499 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4:junit4]   2> 7168 T499 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4:junit4]   2> 7169 T499 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4:junit4]   2> 7169 T499 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4:junit4]   2> 7170 T499 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
> [junit4:junit4]   2> 7193 T499 oasc.CoreContainer.load Registering Log Listener
> [junit4:junit4]   2> 7212 T499 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:43537/solr
> [junit4:junit4]   2> 7213 T499 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4:junit4]   2> 7213 T499 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:43537 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@499a601e
> [junit4:junit4]   2> 7215 T499 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4]   2> 7215 T552 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:43537. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4]   2> 7216 T552 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:43537, initiating session
> [junit4:junit4]   2> 7216 T501 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:59693
> [junit4:junit4]   2> 7217 T501 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:59693
> [junit4:junit4]   2> 7219 T503 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca1e958bc0007 with negotiated timeout 20000 for client /140.211.11.196:59693
> [junit4:junit4]   2> 7219 T552 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:43537, sessionid = 0x13ca1e958bc0007, negotiated timeout = 20000
> [junit4:junit4]   2> 7219 T553 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@499a601e name:ZooKeeperConnection Watcher:127.0.0.1:43537 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4]   2> 7220 T499 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4]   2> 7221 T504 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca1e958bc0007
> [junit4:junit4]   2> 7232 T553 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4]   2> 7232 T501 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:59693 which had sessionid 0x13ca1e958bc0007
> [junit4:junit4]   2> 7232 T499 oaz.ZooKeeper.close Session: 0x13ca1e958bc0007 closed
> [junit4:junit4]   2> 7233 T499 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
> [junit4:junit4]   2> 7247 T499 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:43537/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@11b5f02e
> [junit4:junit4]   2> 7248 T499 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4]   2> 7248 T554 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:43537. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4]   2> 7249 T554 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:43537, initiating session
> [junit4:junit4]   2> 7249 T501 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:28727
> [junit4:junit4]   2> 7249 T501 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:28727
> [junit4:junit4]   2> 7251 T503 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca1e958bc0008 with negotiated timeout 20000 for client /140.211.11.196:28727
> [junit4:junit4]   2> 7251 T554 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:43537, sessionid = 0x13ca1e958bc0008, negotiated timeout = 20000
> [junit4:junit4]   2> 7251 T555 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@11b5f02e name:ZooKeeperConnection Watcher:127.0.0.1:43537/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4]   2> 7251 T499 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4]   2> 7253 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0008 type:create cxid:0x1 zxid:0x6c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 7254 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0008 type:create cxid:0x2 zxid:0x6d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 7256 T499 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39198_lt_
> [junit4:junit4]   2> 7257 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0008 type:delete cxid:0x4 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:39198_lt_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:39198_lt_
> [junit4:junit4]   2> 7265 T499 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:39198_lt_
> [junit4:junit4]   2> 7268 T521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
> [junit4:junit4]   2> 7268 T527 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
> [junit4:junit4]   2> 7268 T540 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4:junit4]   2> 7269 T540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
> [junit4:junit4]   2> 7270 T521 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4:junit4]   2> 7271 T527 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4:junit4]   2> 7273 T499 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
> [junit4:junit4]   2> 7281 T556 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359926096841/collection1
> [junit4:junit4]   2> 7281 T556 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
> [junit4:junit4]   2> 7282 T556 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4:junit4]   2> 7282 T556 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
> [junit4:junit4]   2> 7284 T556 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359926096841/collection1/'
> [junit4:junit4]   2> 7285 T556 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359926096841/collection1/lib/classes/' to classloader
> [junit4:junit4]   2> 7286 T556 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359926096841/collection1/lib/README' to classloader
> [junit4:junit4]   2> 7339 T556 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4:junit4]   2> 7411 T556 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4:junit4]   2> 7512 T556 oass.IndexSchema.readSchema Reading Solr Schema
> [junit4:junit4]   2> 7518 T556 oass.IndexSchema.readSchema Schema name=test
> [junit4:junit4]   2> 8112 T556 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4:junit4]   2> 8120 T556 oass.IndexSchema.readSchema default search field in schema is text
> [junit4:junit4]   2> 8123 T556 oass.IndexSchema.readSchema unique key field: id
> [junit4:junit4]   2> 8140 T556 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4]   2> 8144 T556 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4]   2> 8147 T556 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4]   2> 8149 T556 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4]   2> 8150 T556 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359926096841/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty2/
> [junit4:junit4]   2> 8150 T556 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@36298619
> [junit4:junit4]   2> 8150 T556 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
> [junit4:junit4]   2> 8151 T556 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty2 forceNew:false
> [junit4:junit4]   2> 8151 T556 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty2
> [junit4:junit4]   2> 8152 T556 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty2/index/
> [junit4:junit4]   2> 8152 T556 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty2/index' doesn't exist. Creating new index...
> [junit4:junit4]   2> 8153 T556 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty2/index forceNew:false
> [junit4:junit4]   2> 8158 T556 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@254256b1; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4]   2> 8158 T556 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
> [junit4:junit4]   2> 8159 T556 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty2/index
> [junit4:junit4]   2> 8159 T556 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
> [junit4:junit4]   2> 8161 T556 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4:junit4]   2> 8161 T556 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
> [junit4:junit4]   2> 8161 T556 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4:junit4]   2> 8161 T556 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
> [junit4:junit4]   2> 8162 T556 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4:junit4]   2> 8162 T556 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4:junit4]   2> 8162 T556 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4:junit4]   2> 8163 T556 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4:junit4]   2> 8163 T556 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4:junit4]   2> 8164 T556 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
> [junit4:junit4]   2> 8164 T556 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
> [junit4:junit4]   2> 8164 T556 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4:junit4]   2> 8165 T556 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
> [junit4:junit4]   2> 8165 T556 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
> [junit4:junit4]   2> 8165 T556 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
> [junit4:junit4]   2> 8166 T556 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4:junit4]   2> 8166 T556 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 8167 T556 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 8167 T556 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 8167 T556 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 8168 T556 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 8168 T556 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 8169 T556 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 8169 T556 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 8169 T556 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
> [junit4:junit4]   2> 8170 T556 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
> [junit4:junit4]   2> 8178 T556 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4:junit4]   2> 8181 T556 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
> [junit4:junit4]   2> 8182 T556 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
> [junit4:junit4]   2> 8184 T556 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty2
> [junit4:junit4]   2> 8185 T556 oass.SolrIndexSearcher.<init> Opening Searcher@550b9e33 main
> [junit4:junit4]   2> 8186 T556 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4:junit4]   2> 8186 T556 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4:junit4]   2> 8187 T556 oashc.SpellCheckComponent.inform Initializing spell checkers
> [junit4:junit4]   2> 8192 T556 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
> [junit4:junit4]   2> 8243 T557 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@550b9e33 main{StandardDirectoryReader(segments_1:1)}
> [junit4:junit4]   2> 8247 T556 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4]   2> 8247 T556 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4]   2> 8406 T522 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 8406 T522 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4]   2> 	  "operation":"state",
> [junit4:junit4]   2> 	  "numShards":"2",
> [junit4:junit4]   2> 	  "shard":"shard1",
> [junit4:junit4]   2> 	  "roles":null,
> [junit4:junit4]   2> 	  "state":"active",
> [junit4:junit4]   2> 	  "core":"collection1",
> [junit4:junit4]   2> 	  "collection":"collection1",
> [junit4:junit4]   2> 	  "node_name":"127.0.0.1:59558_lt_",
> [junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:59558/lt_"}
> [junit4:junit4]   2> 8411 T522 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4]   2> 	  "operation":"state",
> [junit4:junit4]   2> 	  "numShards":"2",
> [junit4:junit4]   2> 	  "shard":null,
> [junit4:junit4]   2> 	  "roles":null,
> [junit4:junit4]   2> 	  "state":"down",
> [junit4:junit4]   2> 	  "core":"collection1",
> [junit4:junit4]   2> 	  "collection":"collection1",
> [junit4:junit4]   2> 	  "node_name":"127.0.0.1:39198_lt_",
> [junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:39198/lt_"}
> [junit4:junit4]   2> 8411 T522 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
> [junit4:junit4]   2> 8411 T522 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
> [junit4:junit4]   2> 8423 T540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4]   2> 8423 T521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4]   2> 8423 T555 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4]   2> 8423 T527 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4]   2> 9250 T556 oasc.CoreContainer.registerCore registering core: collection1
> [junit4:junit4]   2> 9250 T556 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:39198/lt_ collection:collection1 shard:shard2
> [junit4:junit4]   2> 9251 T556 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
> [junit4:junit4]   2> 9275 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0008 type:delete cxid:0x42 zxid:0x7c txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
> [junit4:junit4]   2> 9281 T556 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
> [junit4:junit4]   2> 9281 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0008 type:create cxid:0x43 zxid:0x7d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 9284 T556 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
> [junit4:junit4]   2> 9285 T556 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
> [junit4:junit4]   2> 9285 T556 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:39198/lt_/collection1/
> [junit4:junit4]   2> 9285 T556 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
> [junit4:junit4]   2> 9285 T556 oasc.SyncStrategy.syncToMe http://127.0.0.1:39198/lt_/collection1/ has no replicas
> [junit4:junit4]   2> 9286 T556 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:39198/lt_/collection1/
> [junit4:junit4]   2> 9286 T556 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
> [junit4:junit4]   2> 9298 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0008 type:create cxid:0x4c zxid:0x80 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 9937 T522 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 9976 T521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4]   2> 9977 T555 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4]   2> 9977 T527 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4]   2> 9976 T540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4]   2> 9980 T556 oasc.ZkController.register We are http://127.0.0.1:39198/lt_/collection1/ and leader is http://127.0.0.1:39198/lt_/collection1/
> [junit4:junit4]   2> 9980 T556 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:39198/lt_
> [junit4:junit4]   2> 9980 T556 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
> [junit4:junit4]   2> 9980 T556 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4:junit4]   2> 9981 T556 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4]   2> 9986 T556 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 9993 T499 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
> [junit4:junit4]   2> 9994 T499 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4:junit4]   2> 9995 T499 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4]   2> 10200 T499 oejs.Server.doStart jetty-8.1.8.v20121106
> [junit4:junit4]   2> 10204 T499 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:48808
> [junit4:junit4]   2> 10204 T499 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4:junit4]   2> 10205 T499 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4:junit4]   2> 10205 T499 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359926099918
> [junit4:junit4]   2> 10206 T499 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359926099918/solr.xml
> [junit4:junit4]   2> 10206 T499 oasc.CoreContainer.<init> New CoreContainer 1199416839
> [junit4:junit4]   2> 10207 T499 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359926099918/'
> [junit4:junit4]   2> 10207 T499 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359926099918/'
> [junit4:junit4]   2> 10262 T499 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
> [junit4:junit4]   2> 10263 T499 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4:junit4]   2> 10263 T499 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4:junit4]   2> 10264 T499 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4:junit4]   2> 10264 T499 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4:junit4]   2> 10264 T499 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4:junit4]   2> 10265 T499 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4:junit4]   2> 10265 T499 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4:junit4]   2> 10266 T499 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4:junit4]   2> 10266 T499 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
> [junit4:junit4]   2> 10291 T499 oasc.CoreContainer.load Registering Log Listener
> [junit4:junit4]   2> 10313 T499 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:43537/solr
> [junit4:junit4]   2> 10314 T499 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4:junit4]   2> 10314 T499 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:43537 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@15f0b12d
> [junit4:junit4]   2> 10315 T499 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4]   2> 10316 T567 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:43537. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4]   2> 10317 T567 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:43537, initiating session
> [junit4:junit4]   2> 10317 T501 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:49643
> [junit4:junit4]   2> 10318 T501 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:49643
> [junit4:junit4]   2> 10319 T503 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca1e958bc0009 with negotiated timeout 20000 for client /140.211.11.196:49643
> [junit4:junit4]   2> 10320 T567 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:43537, sessionid = 0x13ca1e958bc0009, negotiated timeout = 20000
> [junit4:junit4]   2> 10320 T568 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@15f0b12d name:ZooKeeperConnection Watcher:127.0.0.1:43537 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4]   2> 10320 T499 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4]   2> 10322 T504 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca1e958bc0009
> [junit4:junit4]   2> 10323 T568 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4]   2> 10323 T501 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:49643 which had sessionid 0x13ca1e958bc0009
> [junit4:junit4]   2> 10323 T499 oaz.ZooKeeper.close Session: 0x13ca1e958bc0009 closed
> [junit4:junit4]   2> 10324 T499 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
> [junit4:junit4]   2> 10337 T499 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:43537/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@39cc1709
> [junit4:junit4]   2> 10338 T499 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4]   2> 10339 T569 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:43537. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4]   2> 10339 T569 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:43537, initiating session
> [junit4:junit4]   2> 10340 T501 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:19439
> [junit4:junit4]   2> 10340 T501 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:19439
> [junit4:junit4]   2> 10348 T503 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca1e958bc000a with negotiated timeout 20000 for client /140.211.11.196:19439
> [junit4:junit4]   2> 10348 T569 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:43537, sessionid = 0x13ca1e958bc000a, negotiated timeout = 20000
> [junit4:junit4]   2> 10348 T570 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@39cc1709 name:ZooKeeperConnection Watcher:127.0.0.1:43537/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4]   2> 10349 T499 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4]   2> 10350 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc000a type:create cxid:0x1 zxid:0x8d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 10351 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc000a type:create cxid:0x2 zxid:0x8e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 10353 T499 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48808_lt_
> [junit4:junit4]   2> 10354 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc000a type:delete cxid:0x4 zxid:0x8f txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:48808_lt_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:48808_lt_
> [junit4:junit4]   2> 10362 T499 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:48808_lt_
> [junit4:junit4]   2> 10364 T521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
> [junit4:junit4]   2> 10365 T527 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
> [junit4:junit4]   2> 10365 T555 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4:junit4]   2> 10365 T555 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4]   2> 10366 T540 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4:junit4]   2> 10366 T540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4]   2> 10367 T521 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4:junit4]   2> 10367 T527 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4:junit4]   2> 10371 T499 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
> [junit4:junit4]   2> 10380 T571 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359926099918/collection1
> [junit4:junit4]   2> 10380 T571 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
> [junit4:junit4]   2> 10381 T571 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4:junit4]   2> 10381 T571 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
> [junit4:junit4]   2> 10383 T571 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359926099918/collection1/'
> [junit4:junit4]   2> 10384 T571 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359926099918/collection1/lib/classes/' to classloader
> [junit4:junit4]   2> 10385 T571 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359926099918/collection1/lib/README' to classloader
> [junit4:junit4]   2> 10440 T571 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4:junit4]   2> 10526 T571 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4:junit4]   2> 10627 T571 oass.IndexSchema.readSchema Reading Solr Schema
> [junit4:junit4]   2> 10634 T571 oass.IndexSchema.readSchema Schema name=test
> [junit4:junit4]   2> 11264 T571 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4:junit4]   2> 11289 T571 oass.IndexSchema.readSchema default search field in schema is text
> [junit4:junit4]   2> 11293 T571 oass.IndexSchema.readSchema unique key field: id
> [junit4:junit4]   2> 11312 T571 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4]   2> 11317 T571 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4]   2> 11322 T571 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4]   2> 11323 T571 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4]   2> 11325 T571 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359926099918/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3/
> [junit4:junit4]   2> 11325 T571 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@36298619
> [junit4:junit4]   2> 11326 T571 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
> [junit4:junit4]   2> 11327 T571 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3 forceNew:false
> [junit4:junit4]   2> 11327 T571 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3
> [junit4:junit4]   2> 11328 T571 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3/index/
> [junit4:junit4]   2> 11328 T571 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3/index' doesn't exist. Creating new index...
> [junit4:junit4]   2> 11329 T571 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3/index forceNew:false
> [junit4:junit4]   2> 11335 T571 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@61c28a10; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4]   2> 11336 T571 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
> [junit4:junit4]   2> 11337 T571 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3/index
> [junit4:junit4]   2> 11337 T571 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
> [junit4:junit4]   2> 11340 T571 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4:junit4]   2> 11340 T571 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
> [junit4:junit4]   2> 11340 T571 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4:junit4]   2> 11341 T571 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
> [junit4:junit4]   2> 11341 T571 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4:junit4]   2> 11342 T571 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4:junit4]   2> 11342 T571 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4:junit4]   2> 11343 T571 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4:junit4]   2> 11343 T571 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4:junit4]   2> 11344 T571 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
> [junit4:junit4]   2> 11345 T571 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
> [junit4:junit4]   2> 11345 T571 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4:junit4]   2> 11346 T571 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
> [junit4:junit4]   2> 11346 T571 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
> [junit4:junit4]   2> 11346 T571 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
> [junit4:junit4]   2> 11347 T571 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4:junit4]   2> 11348 T571 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 11348 T571 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 11349 T571 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 11349 T571 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 11350 T571 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 11351 T571 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 11351 T571 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 11352 T571 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 11353 T571 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
> [junit4:junit4]   2> 11353 T571 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
> [junit4:junit4]   2> 11366 T571 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4:junit4]   2> 11371 T571 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
> [junit4:junit4]   2> 11372 T571 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
> [junit4:junit4]   2> 11375 T571 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3
> [junit4:junit4]   2> 11376 T571 oass.SolrIndexSearcher.<init> Opening Searcher@5b2f54cd main
> [junit4:junit4]   2> 11377 T571 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4:junit4]   2> 11378 T571 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4:junit4]   2> 11378 T571 oashc.SpellCheckComponent.inform Initializing spell checkers
> [junit4:junit4]   2> 11385 T571 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
> [junit4:junit4]   2> 11443 T572 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5b2f54cd main{StandardDirectoryReader(segments_1:1)}
> [junit4:junit4]   2> 11450 T571 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4]   2> 11450 T571 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4]   2> 11496 T522 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 11497 T522 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4]   2> 	  "operation":"state",
> [junit4:junit4]   2> 	  "numShards":"2",
> [junit4:junit4]   2> 	  "shard":"shard2",
> [junit4:junit4]   2> 	  "roles":null,
> [junit4:junit4]   2> 	  "state":"active",
> [junit4:junit4]   2> 	  "core":"collection1",
> [junit4:junit4]   2> 	  "collection":"collection1",
> [junit4:junit4]   2> 	  "node_name":"127.0.0.1:39198_lt_",
> [junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:39198/lt_"}
> [junit4:junit4]   2> 11502 T522 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4]   2> 	  "operation":"state",
> [junit4:junit4]   2> 	  "numShards":"2",
> [junit4:junit4]   2> 	  "shard":null,
> [junit4:junit4]   2> 	  "roles":null,
> [junit4:junit4]   2> 	  "state":"down",
> [junit4:junit4]   2> 	  "core":"collection1",
> [junit4:junit4]   2> 	  "collection":"collection1",
> [junit4:junit4]   2> 	  "node_name":"127.0.0.1:48808_lt_",
> [junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:48808/lt_"}
> [junit4:junit4]   2> 11503 T522 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
> [junit4:junit4]   2> 11503 T522 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
> [junit4:junit4]   2> 11514 T570 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4]   2> 11514 T540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4]   2> 11514 T527 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4]   2> 11514 T555 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4]   2> 11514 T521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4]   2> 12453 T571 oasc.CoreContainer.registerCore registering core: collection1
> [junit4:junit4]   2> 12453 T571 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:48808/lt_ collection:collection1 shard:shard1
> [junit4:junit4]   2> 12483 T571 oasc.ZkController.register We are http://127.0.0.1:48808/lt_/collection1/ and leader is http://127.0.0.1:59558/lt_/collection1/
> [junit4:junit4]   2> 12484 T571 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:48808/lt_
> [junit4:junit4]   2> 12484 T571 oasc.ZkController.checkRecovery Core needs to recover:collection1
> [junit4:junit4]   2> 12484 T571 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
> [junit4:junit4]   2> ASYNC  NEW_CORE C67 name=collection1 org.apache.solr.core.SolrCore@6fbe4c22 url=http://127.0.0.1:48808/lt_/collection1 node=127.0.0.1:48808_lt_ C67_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:48808_lt_, base_url=http://127.0.0.1:48808/lt_}
> [junit4:junit4]   2> 12491 T573 C67 P48808 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
> [junit4:junit4]   2> 12491 T571 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 12492 T573 C67 P48808 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
> [junit4:junit4]   2> 12493 T573 C67 P48808 oasc.ZkController.publish publishing core=collection1 state=recovering
> [junit4:junit4]   2> 12493 T573 C67 P48808 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4]   2> 12493 T499 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
> [junit4:junit4]   2> 12494 T499 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4:junit4]   2> 12494 T499 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4]   2> 12495 T573 C67 P48808 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4]   2> 12691 T499 oejs.Server.doStart jetty-8.1.8.v20121106
> [junit4:junit4]   2> 12694 T499 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:56235
> [junit4:junit4]   2> 12695 T499 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4:junit4]   2> 12695 T499 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4:junit4]   2> 12696 T499 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359926102423
> [junit4:junit4]   2> 12696 T499 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359926102423/solr.xml
> [junit4:junit4]   2> 12696 T499 oasc.CoreContainer.<init> New CoreContainer 8126726
> [junit4:junit4]   2> 12697 T499 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359926102423/'
> [junit4:junit4]   2> 12697 T499 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359926102423/'
> [junit4:junit4]   2> 12752 T499 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
> [junit4:junit4]   2> 12753 T499 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4:junit4]   2> 12753 T499 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4:junit4]   2> 12754 T499 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4:junit4]   2> 12754 T499 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4:junit4]   2> 12754 T499 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4:junit4]   2> 12755 T499 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4:junit4]   2> 12755 T499 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4:junit4]   2> 12756 T499 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4:junit4]   2> 12756 T499 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
> [junit4:junit4]   2> 12780 T499 oasc.CoreContainer.load Registering Log Listener
> [junit4:junit4]   2> 12799 T499 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:43537/solr
> [junit4:junit4]   2> 12800 T499 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4:junit4]   2> 12800 T499 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:43537 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@6678bc7
> [junit4:junit4]   2> 12801 T499 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4]   2> 12802 T583 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:43537. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4]   2> 12802 T583 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:43537, initiating session
> [junit4:junit4]   2> 12803 T501 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:45364
> [junit4:junit4]   2> 12803 T501 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:45364
> [junit4:junit4]   2> 12805 T503 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca1e958bc000b with negotiated timeout 20000 for client /140.211.11.196:45364
> [junit4:junit4]   2> 12805 T583 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:43537, sessionid = 0x13ca1e958bc000b, negotiated timeout = 20000
> [junit4:junit4]   2> 12806 T584 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6678bc7 name:ZooKeeperConnection Watcher:127.0.0.1:43537 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4]   2> 12806 T499 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4]   2> 12807 T504 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca1e958bc000b
> [junit4:junit4]   2> 12819 T584 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4]   2> 12819 T501 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:45364 which had sessionid 0x13ca1e958bc000b
> [junit4:junit4]   2> 12819 T499 oaz.ZooKeeper.close Session: 0x13ca1e958bc000b closed
> [junit4:junit4]   2> 12820 T499 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
> [junit4:junit4]   2> 12834 T499 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:43537/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@48a02f43
> [junit4:junit4]   2> 12835 T499 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4]   2> 12835 T585 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:43537. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4]   2> 12836 T585 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:43537, initiating session
> [junit4:junit4]   2> 12836 T501 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:34033
> [junit4:junit4]   2> 12836 T501 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:34033
> [junit4:junit4]   2> 12838 T503 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca1e958bc000c with negotiated timeout 20000 for client /140.211.11.196:34033
> [junit4:junit4]   2> 12838 T585 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:43537, sessionid = 0x13ca1e958bc000c, negotiated timeout = 20000
> [junit4:junit4]   2> 12838 T586 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@48a02f43 name:ZooKeeperConnection Watcher:127.0.0.1:43537/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4]   2> 12838 T499 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4]   2> 12840 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc000c type:create cxid:0x1 zxid:0x9f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 12841 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc000c type:create cxid:0x2 zxid:0xa0 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 12843 T499 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56235_lt_
> [junit4:junit4]   2> 12844 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc000c type:delete cxid:0x4 zxid:0xa1 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:56235_lt_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:56235_lt_
> [junit4:junit4]   2> 12851 T499 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:56235_lt_
> [junit4:junit4]   2> 12854 T570 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4]   2> 12854 T521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4]   2> 12854 T527 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4]   2> 12855 T555 oascc.ZkStateReader$3.process Updating live nodes... (5)
> [junit4:junit4]   2> 12855 T540 oascc.ZkStateReader$3.process Updating live nodes... (5)
> [junit4:junit4]   2> 12855 T555 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 12855 T540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 12857 T521 oascc.ZkStateReader$3.process Updating live nodes... (5)
> [junit4:junit4]   2> 12858 T570 oascc.ZkStateReader$3.process Updating live nodes... (5)
> [junit4:junit4]   2> 12858 T527 oascc.ZkStateReader$3.process Updating live nodes... (5)
> [junit4:junit4]   2> 12860 T499 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
> [junit4:junit4]   2> 12867 T587 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359926102423/collection1
> [junit4:junit4]   2> 12867 T587 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
> [junit4:junit4]   2> 12868 T587 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4:junit4]   2> 12868 T587 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
> [junit4:junit4]   2> 12870 T587 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359926102423/collection1/'
> [junit4:junit4]   2> 12871 T587 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359926102423/collection1/lib/README' to classloader
> [junit4:junit4]   2> 12872 T587 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359926102423/collection1/lib/classes/' to classloader
> [junit4:junit4]   2> 12925 T587 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4:junit4]   2> 12997 T587 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4:junit4]   2> 13029 T522 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 13030 T522 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4]   2> 	  "operation":"state",
> [junit4:junit4]   2> 	  "numShards":"2",
> [junit4:junit4]   2> 	  "shard":"shard1",
> [junit4:junit4]   2> 	  "roles":null,
> [junit4:junit4]   2> 	  "state":"recovering",
> [junit4:junit4]   2> 	  "core":"collection1",
> [junit4:junit4]   2> 	  "collection":"collection1",
> [junit4:junit4]   2> 	  "node_name":"127.0.0.1:48808_lt_",
> [junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:48808/lt_"}
> [junit4:junit4]   2> 13036 T570 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 13036 T521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 13036 T555 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 13036 T527 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 13036 T540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 13098 T587 oass.IndexSchema.readSchema Reading Solr Schema
> [junit4:junit4]   2> 13098 T586 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 13105 T587 oass.IndexSchema.readSchema Schema name=test
> [junit4:junit4]   2> 13728 T587 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4:junit4]   2> 13740 T587 oass.IndexSchema.readSchema default search field in schema is text
> [junit4:junit4]   2> 13744 T587 oass.IndexSchema.readSchema unique key field: id
> [junit4:junit4]   2> 13763 T587 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4]   2> 13768 T587 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4]   2> 13773 T587 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4]   2> 13774 T587 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4]   2> 13776 T587 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359926102423/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4/
> [junit4:junit4]   2> 13776 T587 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@36298619
> [junit4:junit4]   2> 13777 T587 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
> [junit4:junit4]   2> 13778 T587 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4 forceNew:false
> [junit4:junit4]   2> 13779 T587 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4
> [junit4:junit4]   2> 13779 T587 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4/index/
> [junit4:junit4]   2> 13779 T587 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4/index' doesn't exist. Creating new index...
> [junit4:junit4]   2> 13781 T587 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4/index forceNew:false
> [junit4:junit4]   2> 13787 T587 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3df9fbf9; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4]   2> 13788 T587 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
> [junit4:junit4]   2> 13789 T587 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4/index
> [junit4:junit4]   2> 13789 T587 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
> [junit4:junit4]   2> 13791 T587 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4:junit4]   2> 13792 T587 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
> [junit4:junit4]   2> 13792 T587 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4:junit4]   2> 13792 T587 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
> [junit4:junit4]   2> 13793 T587 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4:junit4]   2> 13793 T587 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4:junit4]   2> 13794 T587 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4:junit4]   2> 13794 T587 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4:junit4]   2> 13795 T587 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4:junit4]   2> 13796 T587 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
> [junit4:junit4]   2> 13796 T587 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
> [junit4:junit4]   2> 13797 T587 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4:junit4]   2> 13798 T587 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
> [junit4:junit4]   2> 13798 T587 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
> [junit4:junit4]   2> 13798 T587 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
> [junit4:junit4]   2> 13799 T587 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4:junit4]   2> 13799 T587 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 13800 T587 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 13801 T587 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 13801 T587 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 13802 T587 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 13803 T587 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 13803 T587 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 13804 T587 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 13804 T587 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
> [junit4:junit4]   2> 13805 T587 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
> [junit4:junit4]   2> 13818 T587 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4:junit4]   2> 13823 T587 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
> [junit4:junit4]   2> 13825 T587 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
> [junit4:junit4]   2> 13827 T587 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4
> [junit4:junit4]   2> 13829 T587 oass.SolrIndexSearcher.<init> Opening Searcher@7ec4e721 main
> [junit4:junit4]   2> 13830 T587 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4:junit4]   2> 13830 T587 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4:junit4]   2> 13831 T587 oashc.SpellCheckComponent.inform Initializing spell checkers
> [junit4:junit4]   2> 13838 T587 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
> [junit4:junit4]   2> 13893 T588 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7ec4e721 main{StandardDirectoryReader(segments_1:1)}
> [junit4:junit4]   2> 13900 T587 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4]   2> 13900 T587 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4]   2> 14548 T522 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 14549 T522 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4]   2> 	  "operation":"state",
> [junit4:junit4]   2> 	  "numShards":"2",
> [junit4:junit4]   2> 	  "shard":null,
> [junit4:junit4]   2> 	  "roles":null,
> [junit4:junit4]   2> 	  "state":"down",
> [junit4:junit4]   2> 	  "core":"collection1",
> [junit4:junit4]   2> 	  "collection":"collection1",
> [junit4:junit4]   2> 	  "node_name":"127.0.0.1:56235_lt_",
> [junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:56235/lt_"}
> [junit4:junit4]   2> 14550 T522 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
> [junit4:junit4]   2> 14550 T522 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
> [junit4:junit4]   2> 14563 T586 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 14563 T527 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 14563 T555 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 14563 T540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 14563 T570 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 14563 T521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 14903 T587 oasc.CoreContainer.registerCore registering core: collection1
> [junit4:junit4]   2> 14903 T587 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:56235/lt_ collection:collection1 shard:shard2
> [junit4:junit4]   2> 14907 T587 oasc.ZkController.register We are http://127.0.0.1:56235/lt_/collection1/ and leader is http://127.0.0.1:39198/lt_/collection1/
> [junit4:junit4]   2> 14907 T587 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:56235/lt_
> [junit4:junit4]   2> 14907 T587 oasc.ZkController.checkRecovery Core needs to recover:collection1
> [junit4:junit4]   2> 14908 T587 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
> [junit4:junit4]   2> ASYNC  NEW_CORE C68 name=collection1 org.apache.solr.core.SolrCore@56dc599f url=http://127.0.0.1:56235/lt_/collection1 node=127.0.0.1:56235_lt_ C68_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:56235_lt_, base_url=http://127.0.0.1:56235/lt_}
> [junit4:junit4]   2> 14909 T589 C68 P56235 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
> [junit4:junit4]   2> 14909 T589 C68 P56235 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
> [junit4:junit4]   2> 14910 T587 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 14910 T589 C68 P56235 oasc.ZkController.publish publishing core=collection1 state=recovering
> [junit4:junit4]   2> 14910 T589 C68 P56235 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4]   2> 14911 T499 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
> [junit4:junit4]   2> 14911 T499 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4:junit4]   2> 14912 T589 C68 P56235 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4]   2> 14912 T499 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4]   2> 14938 T499 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 14940 T499 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
> [junit4:junit4]   2> 14941 T499 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2>  C67_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:48808_lt_, base_url=http://127.0.0.1:48808/lt_}
> [junit4:junit4]   2> 15545 T573 C67 P48808 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:59558/lt_/collection1/ core=collection1 - recoveringAfterStartup=true
> [junit4:junit4]   2> 15555 T573 C67 P48808 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&connTimeout=30000&socketTimeout=30000&retry=false
> [junit4:junit4]   2> 15576 T573 C67 P48808 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:48808/lt_ START replicas=[http://127.0.0.1:59558/lt_/collection1/] nUpdates=100
> [junit4:junit4]   2> 15586 T573 C67 P48808 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
> [junit4:junit4]   2> 15587 T573 C67 P48808 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
> [junit4:junit4]   2> 15587 T573 C67 P48808 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
> [junit4:junit4]   2> 15588 T573 C67 P48808 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
> [junit4:junit4]   2> 15588 T573 C67 P48808 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
> [junit4:junit4]   2> 15588 T573 C67 P48808 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:59558/lt_/collection1/. core=collection1
> [junit4:junit4]   2> 15588 T573 C67 P48808 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4]   2> ASYNC  NEW_CORE C69 name=collection1 org.apache.solr.core.SolrCore@7a561b47 url=http://127.0.0.1:59558/lt_/collection1 node=127.0.0.1:59558_lt_ C69_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:59558_lt_, base_url=http://127.0.0.1:59558/lt_, leader=true}
> [junit4:junit4]   2> 15604 T531 C69 P59558 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
> [junit4:junit4]   2> 15611 T532 C69 P59558 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
> [junit4:junit4]   2> 15612 T532 C69 P59558 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty1
> [junit4:junit4]   2> 15616 T532 C69 P59558 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7bf4515f; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4]   2> 15617 T532 C69 P59558 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
> [junit4:junit4]   2> 15621 T532 C69 P59558 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7bf4515f; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7bf4515f; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
> [junit4:junit4]   2> 15621 T532 C69 P59558 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
> [junit4:junit4]   2> 15622 T532 C69 P59558 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty1
> [junit4:junit4]   2> 15623 T532 C69 P59558 oass.SolrIndexSearcher.<init> Opening Searcher@65c40485 realtime
> [junit4:junit4]   2> 15623 T532 C69 P59558 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4]   2> 15624 T532 C69 P59558 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 14
> [junit4:junit4]   2> 15625 T573 C67 P48808 oash.ReplicationHandler.inform Commits will be reserved for  10000
> [junit4:junit4]   2> 15625 T573 C67 P48808 oascsi.HttpClientUtil.createClient Creating new http client, config:connTimeout=5000&socketTimeout=20000&allowCompression=false&maxConnections=10000&maxConnectionsPerHost=10000
> [junit4:junit4]   2> 15653 T573 C67 P48808 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
> [junit4:junit4]   2> 15656 T533 C69 P59558 oash.ReplicationHandler.inform Commits will be reserved for  10000
> [junit4:junit4]   2> 15657 T533 C69 P59558 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2 
> [junit4:junit4]   2> 15658 T573 C67 P48808 oash.SnapPuller.fetchLatestIndex Master's generation: 2
> [junit4:junit4]   2> 15658 T573 C67 P48808 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
> [junit4:junit4]   2> 15658 T573 C67 P48808 oash.SnapPuller.fetchLatestIndex Starting replication process
> [junit4:junit4]   2> 15662 T533 C69 P59558 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty1
> [junit4:junit4]   2> 15663 T533 C69 P59558 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty1/index
> [junit4:junit4]   2> 15663 T533 C69 P59558 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=2 
> [junit4:junit4]   2> 15664 T573 C67 P48808 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
> [junit4:junit4]   2> 15665 T573 C67 P48808 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3/index.20130203221505568 forceNew:false
> [junit4:junit4]   2> 15666 T573 C67 P48808 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3
> [junit4:junit4]   2> 15667 T573 C67 P48808 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3/index.20130203221505568 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2db7be4d; maxCacheMB=48.0 maxMergeSizeMB=4.0)) fullCopy=true
> [junit4:junit4]   2> 15681 T533 C69 P59558 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=6 
> [junit4:junit4]   2> 15688 T573 C67 P48808 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3/index
> [junit4:junit4]   2> 15689 T573 C67 P48808 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
> [junit4:junit4]   2> 15689 T573 C67 P48808 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130203221505568
> [junit4:junit4]   2> 15691 T573 C67 P48808 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3
> [junit4:junit4]   2> 15692 T573 C67 P48808 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3
> [junit4:junit4]   2> 15692 T573 C67 P48808 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3/index/ new=./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3/index.20130203221505568
> [junit4:junit4]   2> 15696 T573 C67 P48808 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3/index.20130203221505568 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2db7be4d; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
> [junit4:junit4]   2> 15696 T573 C67 P48808 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
> [junit4:junit4]   2> 15698 T573 C67 P48808 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3
> [junit4:junit4]   2> 15703 T573 C67 P48808 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
> [junit4:junit4]   2> 15703 T573 C67 P48808 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
> [junit4:junit4]   2> 15703 T573 C67 P48808 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
> [junit4:junit4]   2> 15704 T573 C67 P48808 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3/index.20130203221505568
> [junit4:junit4]   2> 15705 T573 C67 P48808 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3
> [junit4:junit4]   2> 15709 T573 C67 P48808 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3/index.20130203221505568 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2db7be4d; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
> [junit4:junit4]   2> 15709 T573 C67 P48808 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
> [junit4:junit4]   2> 15710 T573 C67 P48808 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
> [junit4:junit4]   2> 15710 T573 C67 P48808 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3
> [junit4:junit4]   2> 15711 T573 C67 P48808 oass.SolrIndexSearcher.<init> Opening Searcher@3c2d6104 main
> [junit4:junit4]   2> 15717 T572 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3c2d6104 main{StandardDirectoryReader(segments_2:1:nrt)}
> [junit4:junit4]   2> 15718 T572 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3/index
> [junit4:junit4]   2> 15719 T573 C67 P48808 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
> [junit4:junit4]   2> 15776 T573 C67 P48808 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3/index.20130203221505568 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2db7be4d; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3/index.20130203221505568 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2db7be4d; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_3,generation=3,filenames=[segments_3]
> [junit4:junit4]   2> 15777 T573 C67 P48808 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
> [junit4:junit4]   2> 15778 T573 C67 P48808 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3
> [junit4:junit4]   2> 15778 T573 C67 P48808 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4]   2> 15779 T573 C67 P48808 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3/index.20130203221505568
> [junit4:junit4]   2> 15779 T573 C67 P48808 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3/index
> [junit4:junit4]   2> 15779 T573 C67 P48808 oasc.RecoveryStrategy.replay No replay needed. core=collection1
> [junit4:junit4]   2> 15779 T573 C67 P48808 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
> [junit4:junit4]   2> 15780 T573 C67 P48808 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4:junit4]   2> 15780 T573 C67 P48808 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4]   2> 15806 T573 C67 P48808 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
> [junit4:junit4]   2> 15943 T499 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 16069 T522 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 16070 T522 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4]   2> 	  "operation":"state",
> [junit4:junit4]   2> 	  "numShards":"2",
> [junit4:junit4]   2> 	  "shard":"shard2",
> [junit4:junit4]   2> 	  "roles":null,
> [junit4:junit4]   2> 	  "state":"recovering",
> [junit4:junit4]   2> 	  "core":"collection1",
> [junit4:junit4]   2> 	  "collection":"collection1",
> [junit4:junit4]   2> 	  "node_name":"127.0.0.1:56235_lt_",
> [junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:56235/lt_"}
> [junit4:junit4]   2> 16078 T522 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4]   2> 	  "operation":"state",
> [junit4:junit4]   2> 	  "numShards":"2",
> [junit4:junit4]   2> 	  "shard":"shard1",
> [junit4:junit4]   2> 	  "roles":null,
> [junit4:junit4]   2> 	  "state":"active",
> [junit4:junit4]   2> 	  "core":"collection1",
> [junit4:junit4]   2> 	  "collection":"collection1",
> [junit4:junit4]   2> 	  "node_name":"127.0.0.1:48808_lt_",
> [junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:48808/lt_"}
> [junit4:junit4]   2> 16095 T570 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 16095 T540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 16095 T527 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 16095 T555 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 16095 T521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 16095 T586 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 16946 T499 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 17950 T499 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2>  C68_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:56235_lt_, base_url=http://127.0.0.1:56235/lt_}
> [junit4:junit4]   2> 18942 T589 C68 P56235 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:39198/lt_/collection1/ core=collection1 - recoveringAfterStartup=true
> [junit4:junit4]   2> 18942 T589 C68 P56235 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:56235/lt_ START replicas=[http://127.0.0.1:39198/lt_/collection1/] nUpdates=100
> [junit4:junit4]   2> 18943 T589 C68 P56235 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
> [junit4:junit4]   2> 18943 T589 C68 P56235 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
> [junit4:junit4]   2> 18943 T589 C68 P56235 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
> [junit4:junit4]   2> 18943 T589 C68 P56235 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
> [junit4:junit4]   2> 18944 T589 C68 P56235 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
> [junit4:junit4]   2> 18945 T589 C68 P56235 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:39198/lt_/collection1/. core=collection1
> [junit4:junit4]   2> 18945 T589 C68 P56235 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4]   2> 18953 T499 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> ASYNC  NEW_CORE C70 name=collection1 org.apache.solr.core.SolrCore@b2471fb url=http://127.0.0.1:39198/lt_/collection1 node=127.0.0.1:39198_lt_ C70_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:39198_lt_, base_url=http://127.0.0.1:39198/lt_, leader=true}
> [junit4:junit4]   2> 18955 T546 C70 P39198 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
> [junit4:junit4]   2> 18965 T547 C70 P39198 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
> [junit4:junit4]   2> 18966 T547 C70 P39198 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty2
> [junit4:junit4]   2> 18970 T547 C70 P39198 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@254256b1; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4]   2> 18971 T547 C70 P39198 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
> [junit4:junit4]   2> 18982 T547 C70 P39198 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@254256b1; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@254256b1; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
> [junit4:junit4]   2> 18982 T547 C70 P39198 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
> [junit4:junit4]   2> 18984 T547 C70 P39198 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty2
> [junit4:junit4]   2> 18984 T547 C70 P39198 oass.SolrIndexSearcher.<init> Opening Searcher@4aef3675 realtime
> [junit4:junit4]   2> 18985 T547 C70 P39198 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4]   2> 18985 T547 C70 P39198 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 20
> [junit4:junit4]   2> 18986 T589 C68 P56235 oash.ReplicationHandler.inform Commits will be reserved for  10000
> [junit4:junit4]   2> 18986 T589 C68 P56235 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
> [junit4:junit4]   2> 18989 T548 C70 P39198 oash.ReplicationHandler.inform Commits will be reserved for  10000
> [junit4:junit4]   2> 18989 T548 C70 P39198 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1 
> [junit4:junit4]   2> 18990 T589 C68 P56235 oash.SnapPuller.fetchLatestIndex Master's generation: 2
> [junit4:junit4]   2> 18991 T589 C68 P56235 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
> [junit4:junit4]   2> 18991 T589 C68 P56235 oash.SnapPuller.fetchLatestIndex Starting replication process
> [junit4:junit4]   2> 18994 T548 C70 P39198 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty2
> [junit4:junit4]   2> 18994 T548 C70 P39198 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty2/index
> [junit4:junit4]   2> 18995 T548 C70 P39198 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=2 
> [junit4:junit4]   2> 18996 T589 C68 P56235 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
> [junit4:junit4]   2> 18997 T589 C68 P56235 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4/index.20130203221508900 forceNew:false
> [junit4:junit4]   2> 18998 T589 C68 P56235 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4
> [junit4:junit4]   2> 18999 T589 C68 P56235 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4/index.20130203221508900 lockFactory=org.apache.lucene.store.NativeFSLockFactory@f6d976e; maxCacheMB=48.0 maxMergeSizeMB=4.0)) fullCopy=true
> [junit4:junit4]   2> 19002 T548 C70 P39198 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
> [junit4:junit4]   2> 19004 T589 C68 P56235 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4/index
> [junit4:junit4]   2> 19004 T589 C68 P56235 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
> [junit4:junit4]   2> 19005 T589 C68 P56235 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130203221508900
> [junit4:junit4]   2> 19006 T589 C68 P56235 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4
> [junit4:junit4]   2> 19007 T589 C68 P56235 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4
> [junit4:junit4]   2> 19007 T589 C68 P56235 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4/index/ new=./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4/index.20130203221508900
> [junit4:junit4]   2> 19011 T589 C68 P56235 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4/index.20130203221508900 lockFactory=org.apache.lucene.store.NativeFSLockFactory@f6d976e; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
> [junit4:junit4]   2> 19012 T589 C68 P56235 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
> [junit4:junit4]   2> 19014 T589 C68 P56235 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4
> [junit4:junit4]   2> 19014 T589 C68 P56235 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
> [junit4:junit4]   2> 19014 T589 C68 P56235 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
> [junit4:junit4]   2> 19014 T589 C68 P56235 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
> [junit4:junit4]   2> 19015 T589 C68 P56235 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4/index.20130203221508900
> [junit4:junit4]   2> 19016 T589 C68 P56235 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4
> [junit4:junit4]   2> 19021 T589 C68 P56235 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4/index.20130203221508900 lockFactory=org.apache.lucene.store.NativeFSLockFactory@f6d976e; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
> [junit4:junit4]   2> 19021 T589 C68 P56235 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
> [junit4:junit4]   2> 19021 T589 C68 P56235 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
> [junit4:junit4]   2> 19022 T589 C68 P56235 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4
> [junit4:junit4]   2> 19022 T589 C68 P56235 oass.SolrIndexSearcher.<init> Opening Searcher@692cbe60 main
> [junit4:junit4]   2> 19024 T588 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@692cbe60 main{StandardDirectoryReader(segments_2:1:nrt)}
> [junit4:junit4]   2> 19025 T588 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4/index
> [junit4:junit4]   2> 19025 T589 C68 P56235 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
> [junit4:junit4]   2> 19030 T589 C68 P56235 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4/index.20130203221508900 lockFactory=org.apache.lucene.store.NativeFSLockFactory@f6d976e; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4/index.20130203221508900 lockFactory=org.apache.lucene.store.NativeFSLockFactory@f6d976e; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_3,generation=3,filenames=[segments_3]
> [junit4:junit4]   2> 19030 T589 C68 P56235 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
> [junit4:junit4]   2> 19032 T589 C68 P56235 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4
> [junit4:junit4]   2> 19032 T589 C68 P56235 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4]   2> 19032 T589 C68 P56235 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4/index.20130203221508900
> [junit4:junit4]   2> 19033 T589 C68 P56235 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4/index
> [junit4:junit4]   2> 19033 T589 C68 P56235 oasc.RecoveryStrategy.replay No replay needed. core=collection1
> [junit4:junit4]   2> 19033 T589 C68 P56235 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
> [junit4:junit4]   2> 19033 T589 C68 P56235 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4:junit4]   2> 19033 T589 C68 P56235 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4]   2> 19035 T589 C68 P56235 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
> [junit4:junit4]   2> 19115 T522 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 19116 T522 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4]   2> 	  "operation":"state",
> [junit4:junit4]   2> 	  "numShards":"2",
> [junit4:junit4]   2> 	  "shard":"shard2",
> [junit4:junit4]   2> 	  "roles":null,
> [junit4:junit4]   2> 	  "state":"active",
> [junit4:junit4]   2> 	  "core":"collection1",
> [junit4:junit4]   2> 	  "collection":"collection1",
> [junit4:junit4]   2> 	  "node_name":"127.0.0.1:56235_lt_",
> [junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:56235/lt_"}
> [junit4:junit4]   2> 19130 T570 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 19131 T527 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 19130 T555 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 19130 T521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 19130 T586 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 19130 T540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 19956 T499 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 19957 T499 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
> [junit4:junit4]   2> ASYNC  NEW_CORE C71 name=collection1 org.apache.solr.core.SolrCore@6a68c9a1 url=http://127.0.0.1:25591/lt_/collection1 node=127.0.0.1:25591_lt_ C71_STATE=coll:control_collection core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:25591_lt_, base_url=http://127.0.0.1:25591/lt_, leader=true}
> [junit4:junit4]   2> 19971 T511 C71 P25591 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/control/data
> [junit4:junit4]   2> 19975 T511 C71 P25591 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@708ce5ef; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4]   2> 19976 T511 C71 P25591 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
> [junit4:junit4]   2> 19977 T511 C71 P25591 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/control/data
> [junit4:junit4]   2> 19978 T511 C71 P25591 oass.SolrIndexSearcher.<init> Opening Searcher@159adcf5 realtime
> [junit4:junit4]   2> 19979 T511 C71 P25591 /update {wt=javabin&version=2} {deleteByQuery=*:* (-1425985880586190848)} 0 10
> [junit4:junit4]   2> 19995 T549 C70 P39198 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty2
> [junit4:junit4]   2> 20000 T534 C69 P59558 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty1
> [junit4:junit4]   2>  C68_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:56235_lt_, base_url=http://127.0.0.1:56235/lt_}
> [junit4:junit4]   2> 20010 T576 C68 P56235 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4
> [junit4:junit4]   2> 20011 T576 C68 P56235 /update {update.distrib=FROMLEADER&_version_=-1425985880611356672&update.from=http://127.0.0.1:39198/lt_/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1425985880611356672)} 0 3
> [junit4:junit4]   2>  C67_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:48808_lt_, base_url=http://127.0.0.1:48808/lt_}
> [junit4:junit4]   2> 20014 T560 C67 P48808 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3
> [junit4:junit4]   2> 20015 T560 C67 P48808 /update {update.distrib=FROMLEADER&_version_=-1425985880617648128&update.from=http://127.0.0.1:59558/lt_/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1425985880617648128)} 0 3
> [junit4:junit4]   2> 20016 T534 C69 P59558 /update {update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1425985880617648128)} 0 18
> [junit4:junit4]   2> 20017 T549 C70 P39198 /update {wt=javabin&version=2} {deleteByQuery=*:* (-1425985880611356672)} 0 25
> [junit4:junit4]   2> 20027 T511 C71 P25591 /update {wt=javabin&version=2} {add=[1 (1425985880641765376)]} 0 5
> [junit4:junit4]   2> 20054 T560 C67 P48808 /update {distrib.from=http://127.0.0.1:59558/lt_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1425985880659591168)]} 0 6
> [junit4:junit4]   2> 20055 T534 C69 P59558 /update {distrib.from=http://127.0.0.1:39198/lt_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1425985880659591168)]} 0 16
> [junit4:junit4]   2> 20056 T549 C70 P39198 /update {wt=javabin&version=2} {add=[1]} 0 26
> [junit4:junit4]   2> 20062 T511 C71 P25591 /update {wt=javabin&version=2} {add=[2 (1425985880681611264)]} 0 2
> [junit4:junit4]   2> 20079 T576 C68 P56235 /update {distrib.from=http://127.0.0.1:39198/lt_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1425985880689999872)]} 0 5
> [junit4:junit4]   2> 20080 T550 C70 P39198 /update {distrib.from=http://127.0.0.1:59558/lt_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[2 (1425985880689999872)]} 0 13
> [junit4:junit4]   2> 20081 T535 C69 P59558 /update {wt=javabin&version=2} {add=[2]} 0 16
> [junit4:junit4]   2> 20087 T511 C71 P25591 /update {wt=javabin&version=2} {add=[3 (1425985880707825664)]} 0 3
> [junit4:junit4]   2> 20101 T576 C68 P56235 /update {distrib.from=http://127.0.0.1:39198/lt_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[3 (1425985880717262848)]} 0 3
> [junit4:junit4]   2> 20102 T550 C70 P39198 /update {distrib.from=http://127.0.0.1:59558/lt_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[3 (1425985880717262848)]} 0 9
> [junit4:junit4]   2> 20103 T535 C69 P59558 /update {wt=javabin&version=2} {add=[3]} 0 13
> [junit4:junit4]   2> 20109 T511 C71 P25591 /update {wt=javabin&version=2} {add=[4 (1425985880730894336)]} 0 3
> [junit4:junit4]   2> 20124 T560 C67 P48808 /update {distrib.from=http://127.0.0.1:59558/lt_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[4 (1425985880740331520)]} 0 3
> [junit4:junit4]   2> 20125 T534 C69 P59558 /update {distrib.from=http://127.0.0.1:39198/lt_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[4 (1425985880740331520)]} 0 10
> [junit4:junit4]   2> 20126 T549 C70 P39198 /update {wt=javabin&version=2} {add=[4]} 0 14
> [junit4:junit4]   2> 20132 T511 C71 P25591 /update {wt=javabin&version=2} {add=[5 (1425985880755011584)]} 0 3
> [junit4:junit4]   2> 20146 T576 C68 P56235 /update {distrib.from=http://127.0.0.1:39198/lt_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[5 (1425985880764448768)]} 0 3
> [junit4:junit4]   2> 20147 T550 C70 P39198 /update {distrib.from=http://127.0.0.1:59558/lt_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[5 (1425985880764448768)]} 0 9
> [junit4:junit4]   2> 20148 T535 C69 P59558 /update {wt=javabin&version=2} {add=[5]} 0 12
> [junit4:junit4]   2> 20154 T511 C71 P25591 /update {wt=javabin&version=2} {add=[6 (1425985880778080256)]} 0 3
> [junit4:junit4]   2> 20168 T576 C68 P56235 /update {distrib.from=http://127.0.0.1:39198/lt_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[6 (1425985880787517440)]} 0 3
> [junit4:junit4]   2> 20169 T550 C70 P39198 /update {distrib.from=http://127.0.0.1:59558/lt_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[6 (1425985880787517440)]} 0 9
> [junit4:junit4]   2> 20170 T535 C69 P59558 /update {wt=javabin&version=2} {add=[6]} 0 13
> [junit4:junit4]   2> 20176 T511 C71 P25591 /update {wt=javabin&version=2} {add=[7 (1425985880801148928)]} 0 3
> [junit4:junit4]   2> 20189 T576 C68 P56235 /update {distrib.from=http://127.0.0.1:39198/lt_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[7 (1425985880807440384)]} 0 3
> [junit4:junit4]   2> 20190 T549 C70 P39198 /update {wt=javabin&version=2} {add=[7 (1425985880807440384)]} 0 11
> [junit4:junit4]   2> 20196 T511 C71 P25591 /update {wt=javabin&version=2} {add=[8 (1425985880822120448)]} 0 3
> [junit4:junit4]   2> 20211 T560 C67 P48808 /update {distrib.from=http://127.0.0.1:59558/lt_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[8 (1425985880831557632)]} 0 3
> [junit4:junit4]   2> 20212 T534 C69 P59558 /update {distrib.from=http://127.0.0.1:39198/lt_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[8 (1425985880831557632)]} 0 10
> [junit4:junit4]   2> 20213 T549 C70 P39198 /update {wt=javabin&version=2} {add=[8]} 0 14
> [junit4:junit4]   2> 20219 T511 C71 P25591 /update {wt=javabin&version=2} {add=[9 (1425985880846237696)]} 0 3
> [junit4:junit4]   2> 20233 T576 C68 P56235 /update {distrib.from=http://127.0.0.1:39198/lt_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[9 (1425985880855674880)]} 0 3
> [junit4:junit4]   2> 20233 T550 C70 P39198 /update {distrib.from=http://127.0.0.1:59558/lt_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[9 (1425985880855674880)]} 0 8
> [junit4:junit4]   2> 20234 T535 C69 P59558 /update {wt=javabin&version=2} {add=[9]} 0 12
> [junit4:junit4]   2> 20261 T511 C71 P25591 /update {wt=javabin&version=2} {add=[10 (1425985880869306368)]} 0 23
> [junit4:junit4]   2> 20276 T560 C67 P48808 /update {distrib.from=http://127.0.0.1:59558/lt_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10 (1425985880900763648)]} 0 3
> [junit4:junit4]   2> 20276 T534 C69 P59558 /update {distrib.from=http://127.0.0.1:39198/lt_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[10 (1425985880900763648)]} 0 8
> [junit4:junit4]   2> 20277 T549 C70 P39198 /update {wt=javabin&version=2} {add=[10]} 0 12
> [junit4:junit4]   2> 20284 T511 C71 P25591 /update {wt=javabin&version=2} {add=[11 (1425985880913346560)]} 0 4
> [junit4:junit4]   2> 20298 T560 C67 P48808 /update {distrib.from=http://127.0.0.1:59558/lt_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[11 (1425985880923832320)]} 0 3
> [junit4:junit4]   2> 20299 T534 C69 P59558 /update {distrib.from=http://127.0.0.1:39198/lt_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[11 (1425985880923832320)]} 0 9
> [junit4:junit4]   2> 20299 T549 C70 P39198 /update {wt=javabin&version=2} {add=[11]} 0 12
> [junit4:junit4]   2> 20306 T511 C71 P25591 /update {wt=javabin&version=2} {add=[12 (1425985880937463808)]} 0 3
> [junit4:junit4]   2> 20320 T560 C67 P48808 /update {distrib.from=http://127.0.0.1:59558/lt_/collection1/&update.distrib=FROMLEADER&wt=
> 
> [...truncated too long message...]
> 
> rdownChained.java:50)
> [junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
> [junit4:junit4]   2> 	        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
> [junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> [junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
> [junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
> [junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> [junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> [junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
> [junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
> [junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
> [junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
> [junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
> [junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
> [junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
> [junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> [junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
> [junit4:junit4]   2> 	        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
> [junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
> [junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> [junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
> [junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
> [junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> [junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
> [junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> [junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
> [junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
> [junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> [junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
> [junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
> [junit4:junit4]   2> 	  22) Thread[id=572, name=searcherExecutor-469-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
> [junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
> [junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> [junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
> [junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> [junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
> [junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
> [junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> [junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
> [junit4:junit4]   2> 	  23) Thread[id=629, name=searcherExecutor-496-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
> [junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
> [junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> [junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
> [junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> [junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
> [junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
> [junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> [junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
> [junit4:junit4]   2> NOTE: test params are: codec=Lucene42: {n_f1=PostingsFormat(name=SimpleText), foo_b=PostingsFormat(name=MockSep), foo_d=MockVariableIntBlock(baseBlockSize=77), foo_f=PostingsFormat(name=MockSep), n_tl1=PostingsFormat(name=SimpleText), n_d1=MockFixedIntBlock(blockSize=1003), rnd_b=MockVariableIntBlock(baseBlockSize=77), intDefault=MockFixedIntBlock(blockSize=1003), n_td1=PostingsFormat(name=SimpleText), timestamp=MockFixedIntBlock(blockSize=1003), id=MockVariableIntBlock(baseBlockSize=77), range_facet_sl=PostingsFormat(name=MockSep), range_facet_si=PostingsFormat(name=SimpleText), oddField_s=PostingsFormat(name=MockSep), foo_i=PostingsFormat(name=SimpleText), multiDefault=PostingsFormat(name=SimpleText), n_tf1=MockFixedIntBlock(blockSize=1003), n_dt1=PostingsFormat(name=SimpleText), n_ti1=MockVariableIntBlock(baseBlockSize=77), range_facet_l=MockFixedIntBlock(blockSize=1003), text=PostingsFormat(name=MockSep), _version_=MockFixedIntBlock(blockSize=1003), val_i=MockVariableIntBlock(baseBlockSize=77), SubjectTerms_mfacet=MockFixedIntBlock(blockSize=1003), a_t=MockFixedIntBlock(blockSize=1003), n_tdt1=MockFixedIntBlock(blockSize=1003), other_tl1=MockFixedIntBlock(blockSize=1003), n_l1=MockFixedIntBlock(blockSize=1003), a_si=PostingsFormat(name=SimpleText)}, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=ro, timezone=Europe/Malta
> [junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=25,free=255086936,total=470810624
> [junit4:junit4]   2> NOTE: All tests run in this JVM: [QueryElevationComponentTest, SystemInfoHandlerTest, JSONWriterTest, CopyFieldTest, DateFieldTest, TestSuggestSpellingConverter, TestSearchPerf, DirectSolrConnectionTest, NumericFieldsTest, TestReload, SOLR749Test, IndexBasedSpellCheckerTest, SolrCmdDistributorTest, TestFuzzyAnalyzedSuggestions, FastVectorHighlighterTest, PreAnalyzedFieldTest, TestRangeQuery, TestComponentsName, LeaderElectionIntegrationTest, TestDefaultSimilarityFactory, TestPerFieldSimilarity, DirectUpdateHandlerOptimizeTest, NoCacheHeaderTest, TermsComponentTest, DefaultValueUpdateProcessorTest, TestSolrIndexConfig, TestPHPSerializedResponseWriter, SpellCheckComponentTest, TestBinaryResponseWriter, TestIndexingPerformance, MoreLikeThisHandlerTest, FieldAnalysisRequestHandlerTest, TestStressVersions, AlternateDirectoryTest, TestSolr4Spatial, UpdateRequestProcessorFactoryTest, TestPseudoReturnFields, AutoCommitTest, TestNumberUtils, TestFastOutputStream, BasicDistributedZkTest]
> [junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=BasicDistributedZkTest -Dtests.seed=437C1ACFB7351B0A -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ro -Dtests.timezone=Europe/Malta -Dtests.file.encoding=US-ASCII
> [junit4:junit4] ERROR   0.00s J0 | BasicDistributedZkTest (suite) <<<
> [junit4:junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
> [junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([437C1ACFB7351B0A]:0)
> [junit4:junit4] Completed on J0 in 7223.60s, 1 test, 2 errors <<< FAILURES!
> 
> [...truncated 74 lines...]
> BUILD FAILED
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:346: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:39: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build.xml:178: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/common-build.xml:428: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:1176: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:841: There were test failures: 249 suites, 1038 tests, 1 suite-level error, 1 error, 66 ignored (5 assumptions)
> 
> Total time: 144 minutes 57 seconds
> Build step 'Invoke Ant' marked build as failure
> Archiving artifacts
> Recording test results
> Email was triggered for: Failure
> Sending email for trigger: Failure
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[JENKINS] Lucene-Solr-Tests-trunk-java7 - Build # 3705 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-java7/3705/

2 tests failed.
FAILED:  org.apache.solr.cloud.BasicDistributedZkTest.testDistribSearch

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([437C1ACFB7351B0A]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([437C1ACFB7351B0A]:0)




Build Log:
[...truncated 9446 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
[junit4:junit4]   2> 1 T499 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /lt_/
[junit4:junit4]   2> 6 T499 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZkTest-1359926089911
[junit4:junit4]   2> 7 T499 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 8 T500 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 10 T500 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 10 T500 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4]   2> 10 T500 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4]   2> 11 T500 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 11 T500 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4]   2> 109 T499 oasc.ZkTestServer.run start zk server on port:43537
[junit4:junit4]   2> 109 T499 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:43537 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@40f81bb4
[junit4:junit4]   2> 110 T499 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 111 T505 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:43537. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 112 T505 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:43537, initiating session
[junit4:junit4]   2> 112 T501 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:43538
[junit4:junit4]   2> 112 T501 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:43538
[junit4:junit4]   2> 113 T503 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 117 T503 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca1e958bc0000 with negotiated timeout 10000 for client /140.211.11.196:43538
[junit4:junit4]   2> 117 T505 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:43537, sessionid = 0x13ca1e958bc0000, negotiated timeout = 10000
[junit4:junit4]   2> 117 T506 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@40f81bb4 name:ZooKeeperConnection Watcher:127.0.0.1:43537 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 117 T499 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 118 T499 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 121 T504 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca1e958bc0000
[junit4:junit4]   2> 133 T506 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 133 T499 oaz.ZooKeeper.close Session: 0x13ca1e958bc0000 closed
[junit4:junit4]   2> 134 T501 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:43538 which had sessionid 0x13ca1e958bc0000
[junit4:junit4]   2> 134 T499 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:43537/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@70df8d56
[junit4:junit4]   2> 135 T499 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 135 T507 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:43537. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 136 T507 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:43537, initiating session
[junit4:junit4]   2> 136 T501 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:43539
[junit4:junit4]   2> 136 T501 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:43539
[junit4:junit4]   2> 160 T503 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca1e958bc0001 with negotiated timeout 10000 for client /140.211.11.196:43539
[junit4:junit4]   2> 160 T507 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:43537, sessionid = 0x13ca1e958bc0001, negotiated timeout = 10000
[junit4:junit4]   2> 160 T508 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@70df8d56 name:ZooKeeperConnection Watcher:127.0.0.1:43537/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 160 T499 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 161 T499 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 166 T499 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 175 T499 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 178 T499 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 182 T499 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 183 T499 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 196 T499 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 197 T499 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 302 T499 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 302 T499 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 306 T499 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 307 T499 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 310 T499 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 311 T499 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 315 T499 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 315 T499 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 319 T499 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 320 T499 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 323 T499 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 324 T499 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 327 T499 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 328 T499 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 331 T504 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca1e958bc0001
[junit4:junit4]   2> 338 T508 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 338 T501 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:43539 which had sessionid 0x13ca1e958bc0001
[junit4:junit4]   2> 338 T499 oaz.ZooKeeper.close Session: 0x13ca1e958bc0001 closed
[junit4:junit4]   2> 486 T499 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 490 T499 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:25591
[junit4:junit4]   2> 490 T499 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 491 T499 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 491 T499 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359926090243
[junit4:junit4]   2> 492 T499 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359926090243/solr.xml
[junit4:junit4]   2> 492 T499 oasc.CoreContainer.<init> New CoreContainer 2073771471
[junit4:junit4]   2> 492 T499 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359926090243/'
[junit4:junit4]   2> 493 T499 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359926090243/'
[junit4:junit4]   2> 562 T499 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 562 T499 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 563 T499 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 563 T499 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 564 T499 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 564 T499 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 565 T499 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 565 T499 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 566 T499 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 566 T499 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 590 T499 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 610 T499 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:43537/solr
[junit4:junit4]   2> 610 T499 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 611 T499 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:43537 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@3daa432
[junit4:junit4]   2> 612 T499 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 612 T518 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:43537. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 613 T518 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:43537, initiating session
[junit4:junit4]   2> 614 T501 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:55204
[junit4:junit4]   2> 614 T501 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:55204
[junit4:junit4]   2> 619 T503 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca1e958bc0002 with negotiated timeout 20000 for client /140.211.11.196:55204
[junit4:junit4]   2> 619 T518 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:43537, sessionid = 0x13ca1e958bc0002, negotiated timeout = 20000
[junit4:junit4]   2> 619 T519 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3daa432 name:ZooKeeperConnection Watcher:127.0.0.1:43537 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 619 T499 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 621 T504 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca1e958bc0002
[junit4:junit4]   2> 622 T519 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 622 T501 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:55204 which had sessionid 0x13ca1e958bc0002
[junit4:junit4]   2> 622 T499 oaz.ZooKeeper.close Session: 0x13ca1e958bc0002 closed
[junit4:junit4]   2> 623 T499 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 637 T499 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:43537/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@7eb5d65e
[junit4:junit4]   2> 638 T499 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 638 T520 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:43537. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 639 T520 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:43537, initiating session
[junit4:junit4]   2> 639 T501 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:45054
[junit4:junit4]   2> 639 T501 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:45054
[junit4:junit4]   2> 641 T503 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca1e958bc0003 with negotiated timeout 20000 for client /140.211.11.196:45054
[junit4:junit4]   2> 641 T520 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:43537, sessionid = 0x13ca1e958bc0003, negotiated timeout = 20000
[junit4:junit4]   2> 641 T521 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7eb5d65e name:ZooKeeperConnection Watcher:127.0.0.1:43537/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 641 T499 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 644 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 652 T499 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 654 T499 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:25591_lt_
[junit4:junit4]   2> 655 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0003 type:delete cxid:0x7 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:25591_lt_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:25591_lt_
[junit4:junit4]   2> 657 T499 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:25591_lt_
[junit4:junit4]   2> 667 T499 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 680 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0003 type:delete cxid:0x15 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 681 T499 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 685 T499 oasc.Overseer.start Overseer (id=89124116228734979-127.0.0.1:25591_lt_-n_0000000000) starting
[junit4:junit4]   2> 685 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0003 type:create cxid:0x1a zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 691 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0003 type:create cxid:0x1b zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 693 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0003 type:create cxid:0x1c zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 699 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0003 type:create cxid:0x1d zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 701 T523 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 702 T499 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 705 T499 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 708 T522 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 720 T524 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359926090243/collection1
[junit4:junit4]   2> 720 T524 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 721 T524 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 722 T524 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 724 T524 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359926090243/collection1/'
[junit4:junit4]   2> 725 T524 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359926090243/collection1/lib/README' to classloader
[junit4:junit4]   2> 726 T524 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359926090243/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 782 T524 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 854 T524 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 955 T524 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 961 T524 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 1569 T524 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1577 T524 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1580 T524 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1596 T524 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1600 T524 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1603 T524 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1605 T524 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1606 T524 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359926090243/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/control/data/
[junit4:junit4]   2> 1606 T524 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@36298619
[junit4:junit4]   2> 1607 T524 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 1630 T524 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/control/data forceNew:false
[junit4:junit4]   2> 1630 T524 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/control/data
[junit4:junit4]   2> 1630 T524 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/control/data/index/
[junit4:junit4]   2> 1631 T524 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1632 T524 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/control/data/index forceNew:false
[junit4:junit4]   2> 1638 T524 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@708ce5ef; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 1638 T524 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 1639 T524 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/control/data/index
[junit4:junit4]   2> 1639 T524 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 1641 T524 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1641 T524 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 1641 T524 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1641 T524 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 1642 T524 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1642 T524 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1642 T524 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1642 T524 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1643 T524 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1643 T524 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 1644 T524 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 1644 T524 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1644 T524 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 1644 T524 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 1645 T524 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 1645 T524 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1645 T524 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1646 T524 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1646 T524 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1646 T524 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1647 T524 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1647 T524 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1647 T524 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1648 T524 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1648 T524 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 1648 T524 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 1657 T524 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1660 T524 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 1661 T524 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 1663 T524 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/control/data
[junit4:junit4]   2> 1664 T524 oass.SolrIndexSearcher.<init> Opening Searcher@4b06504d main
[junit4:junit4]   2> 1665 T524 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1665 T524 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1665 T524 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 1682 T524 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 1728 T525 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4b06504d main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1732 T524 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 1732 T524 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 1732 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0003 type:create cxid:0x50 zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 2212 T522 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2213 T522 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:25591_lt_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:25591/lt_"}
[junit4:junit4]   2> 2213 T522 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4]   2> 2226 T522 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 2226 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0003 type:create cxid:0x58 zxid:0x2c txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 2245 T521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 2741 T524 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 2741 T524 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:25591/lt_ collection:control_collection shard:shard1
[junit4:junit4]   2> 2742 T524 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4]   2> 2759 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0003 type:delete cxid:0x72 zxid:0x36 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4]   2> 2760 T524 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 2760 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0003 type:create cxid:0x73 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2771 T524 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 2771 T524 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 2771 T524 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:25591/lt_/collection1/
[junit4:junit4]   2> 2772 T524 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 2772 T524 oasc.SyncStrategy.syncToMe http://127.0.0.1:25591/lt_/collection1/ has no replicas
[junit4:junit4]   2> 2772 T524 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:25591/lt_/collection1/
[junit4:junit4]   2> 2772 T524 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4]   2> 2777 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0003 type:create cxid:0x7d zxid:0x3b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3751 T522 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3802 T521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 3851 T524 oasc.ZkController.register We are http://127.0.0.1:25591/lt_/collection1/ and leader is http://127.0.0.1:25591/lt_/collection1/
[junit4:junit4]   2> 3851 T524 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:25591/lt_
[junit4:junit4]   2> 3851 T524 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 3851 T524 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 3852 T524 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 3854 T524 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3855 T499 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 3856 T499 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 3857 T499 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 3873 T499 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 3890 T499 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:43537/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@417dbcb6
[junit4:junit4]   2> 3891 T499 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 3892 T526 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:43537. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 3892 T526 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:43537, initiating session
[junit4:junit4]   2> 3893 T501 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:40197
[junit4:junit4]   2> 3893 T501 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:40197
[junit4:junit4]   2> 3895 T503 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca1e958bc0004 with negotiated timeout 10000 for client /140.211.11.196:40197
[junit4:junit4]   2> 3895 T526 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:43537, sessionid = 0x13ca1e958bc0004, negotiated timeout = 10000
[junit4:junit4]   2> 3895 T527 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@417dbcb6 name:ZooKeeperConnection Watcher:127.0.0.1:43537/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 3895 T499 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 3897 T499 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 3901 T499 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 4069 T499 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 4073 T499 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:59558
[junit4:junit4]   2> 4073 T499 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 4074 T499 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 4074 T499 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359926093806
[junit4:junit4]   2> 4075 T499 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359926093806/solr.xml
[junit4:junit4]   2> 4075 T499 oasc.CoreContainer.<init> New CoreContainer 2041002222
[junit4:junit4]   2> 4076 T499 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359926093806/'
[junit4:junit4]   2> 4076 T499 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359926093806/'
[junit4:junit4]   2> 4131 T499 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 4132 T499 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 4132 T499 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 4132 T499 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 4133 T499 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 4133 T499 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 4134 T499 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 4134 T499 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 4135 T499 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 4135 T499 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 4157 T499 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 4176 T499 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:43537/solr
[junit4:junit4]   2> 4176 T499 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 4177 T499 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:43537 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@14c6f4dd
[junit4:junit4]   2> 4178 T499 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4178 T537 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:43537. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4179 T537 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:43537, initiating session
[junit4:junit4]   2> 4180 T501 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:35606
[junit4:junit4]   2> 4180 T501 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:35606
[junit4:junit4]   2> 4182 T503 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca1e958bc0005 with negotiated timeout 20000 for client /140.211.11.196:35606
[junit4:junit4]   2> 4182 T537 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:43537, sessionid = 0x13ca1e958bc0005, negotiated timeout = 20000
[junit4:junit4]   2> 4182 T538 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@14c6f4dd name:ZooKeeperConnection Watcher:127.0.0.1:43537 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4183 T499 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4184 T504 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca1e958bc0005
[junit4:junit4]   2> 4193 T538 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 4193 T501 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:35606 which had sessionid 0x13ca1e958bc0005
[junit4:junit4]   2> 4193 T499 oaz.ZooKeeper.close Session: 0x13ca1e958bc0005 closed
[junit4:junit4]   2> 4194 T499 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 4205 T499 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:43537/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@4d111d50
[junit4:junit4]   2> 4206 T499 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4207 T539 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:43537. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4207 T539 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:43537, initiating session
[junit4:junit4]   2> 4208 T501 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:25987
[junit4:junit4]   2> 4208 T501 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:25987
[junit4:junit4]   2> 4209 T503 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca1e958bc0006 with negotiated timeout 20000 for client /140.211.11.196:25987
[junit4:junit4]   2> 4209 T539 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:43537, sessionid = 0x13ca1e958bc0006, negotiated timeout = 20000
[junit4:junit4]   2> 4210 T540 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4d111d50 name:ZooKeeperConnection Watcher:127.0.0.1:43537/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4210 T499 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4211 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0006 type:create cxid:0x1 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4213 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0006 type:create cxid:0x2 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4214 T499 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59558_lt_
[junit4:junit4]   2> 4215 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0006 type:delete cxid:0x4 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:59558_lt_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:59558_lt_
[junit4:junit4]   2> 4222 T499 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:59558_lt_
[junit4:junit4]   2> 4225 T521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 4226 T527 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 4228 T521 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 4230 T499 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 4238 T541 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359926093806/collection1
[junit4:junit4]   2> 4238 T541 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 4239 T541 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 4239 T541 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 4241 T541 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359926093806/collection1/'
[junit4:junit4]   2> 4242 T541 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359926093806/collection1/lib/README' to classloader
[junit4:junit4]   2> 4243 T541 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359926093806/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 4295 T541 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 4366 T541 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 4467 T541 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 4473 T541 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 5077 T541 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 5084 T541 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 5087 T541 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 5102 T541 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5106 T541 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5109 T541 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5111 T541 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5112 T541 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359926093806/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty1/
[junit4:junit4]   2> 5112 T541 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@36298619
[junit4:junit4]   2> 5112 T541 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 5113 T541 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty1 forceNew:false
[junit4:junit4]   2> 5113 T541 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty1
[junit4:junit4]   2> 5114 T541 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty1/index/
[junit4:junit4]   2> 5114 T541 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 5115 T541 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty1/index forceNew:false
[junit4:junit4]   2> 5119 T541 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7bf4515f; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 5120 T541 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 5120 T541 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty1/index
[junit4:junit4]   2> 5121 T541 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 5122 T541 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 5122 T541 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 5123 T541 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 5123 T541 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 5123 T541 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 5124 T541 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 5124 T541 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 5124 T541 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 5125 T541 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 5125 T541 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 5126 T541 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 5126 T541 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 5126 T541 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 5127 T541 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 5127 T541 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 5127 T541 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 5128 T541 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5128 T541 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5129 T541 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5129 T541 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5129 T541 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5130 T541 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5130 T541 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5131 T541 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5131 T541 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 5132 T541 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 5140 T541 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 5143 T541 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 5144 T541 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 5146 T541 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty1
[junit4:junit4]   2> 5147 T541 oass.SolrIndexSearcher.<init> Opening Searcher@2aa0240c main
[junit4:junit4]   2> 5148 T541 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 5148 T541 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 5148 T541 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 5153 T541 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 5196 T542 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2aa0240c main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 5200 T541 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 5200 T541 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 5317 T522 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 5318 T522 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:25591_lt_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:25591/lt_"}
[junit4:junit4]   2> 5322 T522 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:59558_lt_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:59558/lt_"}
[junit4:junit4]   2> 5322 T522 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4]   2> 5323 T522 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 5334 T540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5334 T521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5334 T527 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6203 T541 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 6203 T541 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:59558/lt_ collection:collection1 shard:shard1
[junit4:junit4]   2> 6204 T541 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 6224 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0006 type:delete cxid:0x43 zxid:0x5a txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 6225 T541 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 6225 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0006 type:create cxid:0x44 zxid:0x5b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6236 T541 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 6236 T541 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 6237 T541 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:59558/lt_/collection1/
[junit4:junit4]   2> 6237 T541 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 6237 T541 oasc.SyncStrategy.syncToMe http://127.0.0.1:59558/lt_/collection1/ has no replicas
[junit4:junit4]   2> 6237 T541 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:59558/lt_/collection1/
[junit4:junit4]   2> 6238 T541 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 6243 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0006 type:create cxid:0x4e zxid:0x5f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6848 T522 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6892 T540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6892 T521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6892 T527 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6916 T541 oasc.ZkController.register We are http://127.0.0.1:59558/lt_/collection1/ and leader is http://127.0.0.1:59558/lt_/collection1/
[junit4:junit4]   2> 6916 T541 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:59558/lt_
[junit4:junit4]   2> 6916 T541 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 6916 T541 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 6917 T541 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 6919 T541 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6920 T499 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 6920 T499 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 6921 T499 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 7104 T499 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 7107 T499 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:39198
[junit4:junit4]   2> 7108 T499 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 7108 T499 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 7109 T499 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359926096841
[junit4:junit4]   2> 7109 T499 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359926096841/solr.xml
[junit4:junit4]   2> 7110 T499 oasc.CoreContainer.<init> New CoreContainer 1233522068
[junit4:junit4]   2> 7110 T499 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359926096841/'
[junit4:junit4]   2> 7111 T499 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359926096841/'
[junit4:junit4]   2> 7165 T499 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 7166 T499 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 7166 T499 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 7167 T499 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 7167 T499 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 7168 T499 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 7168 T499 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 7169 T499 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 7169 T499 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 7170 T499 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 7193 T499 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 7212 T499 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:43537/solr
[junit4:junit4]   2> 7213 T499 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 7213 T499 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:43537 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@499a601e
[junit4:junit4]   2> 7215 T499 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 7215 T552 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:43537. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 7216 T552 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:43537, initiating session
[junit4:junit4]   2> 7216 T501 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:59693
[junit4:junit4]   2> 7217 T501 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:59693
[junit4:junit4]   2> 7219 T503 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca1e958bc0007 with negotiated timeout 20000 for client /140.211.11.196:59693
[junit4:junit4]   2> 7219 T552 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:43537, sessionid = 0x13ca1e958bc0007, negotiated timeout = 20000
[junit4:junit4]   2> 7219 T553 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@499a601e name:ZooKeeperConnection Watcher:127.0.0.1:43537 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 7220 T499 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 7221 T504 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca1e958bc0007
[junit4:junit4]   2> 7232 T553 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 7232 T501 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:59693 which had sessionid 0x13ca1e958bc0007
[junit4:junit4]   2> 7232 T499 oaz.ZooKeeper.close Session: 0x13ca1e958bc0007 closed
[junit4:junit4]   2> 7233 T499 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 7247 T499 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:43537/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@11b5f02e
[junit4:junit4]   2> 7248 T499 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 7248 T554 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:43537. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 7249 T554 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:43537, initiating session
[junit4:junit4]   2> 7249 T501 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:28727
[junit4:junit4]   2> 7249 T501 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:28727
[junit4:junit4]   2> 7251 T503 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca1e958bc0008 with negotiated timeout 20000 for client /140.211.11.196:28727
[junit4:junit4]   2> 7251 T554 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:43537, sessionid = 0x13ca1e958bc0008, negotiated timeout = 20000
[junit4:junit4]   2> 7251 T555 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@11b5f02e name:ZooKeeperConnection Watcher:127.0.0.1:43537/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 7251 T499 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 7253 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0008 type:create cxid:0x1 zxid:0x6c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7254 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0008 type:create cxid:0x2 zxid:0x6d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7256 T499 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39198_lt_
[junit4:junit4]   2> 7257 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0008 type:delete cxid:0x4 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:39198_lt_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:39198_lt_
[junit4:junit4]   2> 7265 T499 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:39198_lt_
[junit4:junit4]   2> 7268 T521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7268 T527 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7268 T540 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7269 T540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 7270 T521 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7271 T527 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7273 T499 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 7281 T556 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359926096841/collection1
[junit4:junit4]   2> 7281 T556 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 7282 T556 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 7282 T556 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 7284 T556 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359926096841/collection1/'
[junit4:junit4]   2> 7285 T556 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359926096841/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 7286 T556 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359926096841/collection1/lib/README' to classloader
[junit4:junit4]   2> 7339 T556 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 7411 T556 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 7512 T556 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 7518 T556 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 8112 T556 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 8120 T556 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 8123 T556 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 8140 T556 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8144 T556 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8147 T556 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8149 T556 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8150 T556 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359926096841/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty2/
[junit4:junit4]   2> 8150 T556 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@36298619
[junit4:junit4]   2> 8150 T556 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 8151 T556 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty2 forceNew:false
[junit4:junit4]   2> 8151 T556 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty2
[junit4:junit4]   2> 8152 T556 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty2/index/
[junit4:junit4]   2> 8152 T556 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 8153 T556 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty2/index forceNew:false
[junit4:junit4]   2> 8158 T556 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@254256b1; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 8158 T556 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 8159 T556 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty2/index
[junit4:junit4]   2> 8159 T556 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 8161 T556 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 8161 T556 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 8161 T556 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 8161 T556 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 8162 T556 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 8162 T556 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 8162 T556 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 8163 T556 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 8163 T556 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 8164 T556 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 8164 T556 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 8164 T556 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 8165 T556 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 8165 T556 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 8165 T556 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 8166 T556 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 8166 T556 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8167 T556 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8167 T556 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8167 T556 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8168 T556 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8168 T556 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8169 T556 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8169 T556 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8169 T556 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 8170 T556 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 8178 T556 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 8181 T556 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 8182 T556 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 8184 T556 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty2
[junit4:junit4]   2> 8185 T556 oass.SolrIndexSearcher.<init> Opening Searcher@550b9e33 main
[junit4:junit4]   2> 8186 T556 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 8186 T556 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 8187 T556 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 8192 T556 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 8243 T557 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@550b9e33 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 8247 T556 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 8247 T556 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 8406 T522 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8406 T522 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:59558_lt_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:59558/lt_"}
[junit4:junit4]   2> 8411 T522 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:39198_lt_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:39198/lt_"}
[junit4:junit4]   2> 8411 T522 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 8411 T522 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 8423 T540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8423 T521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8423 T555 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8423 T527 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9250 T556 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 9250 T556 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:39198/lt_ collection:collection1 shard:shard2
[junit4:junit4]   2> 9251 T556 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4]   2> 9275 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0008 type:delete cxid:0x42 zxid:0x7c txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
[junit4:junit4]   2> 9281 T556 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 9281 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0008 type:create cxid:0x43 zxid:0x7d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9284 T556 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 9285 T556 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 9285 T556 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:39198/lt_/collection1/
[junit4:junit4]   2> 9285 T556 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 9285 T556 oasc.SyncStrategy.syncToMe http://127.0.0.1:39198/lt_/collection1/ has no replicas
[junit4:junit4]   2> 9286 T556 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:39198/lt_/collection1/
[junit4:junit4]   2> 9286 T556 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4]   2> 9298 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc0008 type:create cxid:0x4c zxid:0x80 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9937 T522 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9976 T521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9977 T555 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9977 T527 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9976 T540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9980 T556 oasc.ZkController.register We are http://127.0.0.1:39198/lt_/collection1/ and leader is http://127.0.0.1:39198/lt_/collection1/
[junit4:junit4]   2> 9980 T556 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:39198/lt_
[junit4:junit4]   2> 9980 T556 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 9980 T556 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 9981 T556 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 9986 T556 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9993 T499 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 9994 T499 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 9995 T499 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 10200 T499 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 10204 T499 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:48808
[junit4:junit4]   2> 10204 T499 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 10205 T499 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 10205 T499 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359926099918
[junit4:junit4]   2> 10206 T499 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359926099918/solr.xml
[junit4:junit4]   2> 10206 T499 oasc.CoreContainer.<init> New CoreContainer 1199416839
[junit4:junit4]   2> 10207 T499 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359926099918/'
[junit4:junit4]   2> 10207 T499 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359926099918/'
[junit4:junit4]   2> 10262 T499 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 10263 T499 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 10263 T499 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 10264 T499 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 10264 T499 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 10264 T499 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 10265 T499 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 10265 T499 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 10266 T499 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 10266 T499 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 10291 T499 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 10313 T499 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:43537/solr
[junit4:junit4]   2> 10314 T499 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 10314 T499 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:43537 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@15f0b12d
[junit4:junit4]   2> 10315 T499 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 10316 T567 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:43537. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 10317 T567 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:43537, initiating session
[junit4:junit4]   2> 10317 T501 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:49643
[junit4:junit4]   2> 10318 T501 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:49643
[junit4:junit4]   2> 10319 T503 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca1e958bc0009 with negotiated timeout 20000 for client /140.211.11.196:49643
[junit4:junit4]   2> 10320 T567 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:43537, sessionid = 0x13ca1e958bc0009, negotiated timeout = 20000
[junit4:junit4]   2> 10320 T568 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@15f0b12d name:ZooKeeperConnection Watcher:127.0.0.1:43537 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 10320 T499 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 10322 T504 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca1e958bc0009
[junit4:junit4]   2> 10323 T568 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 10323 T501 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:49643 which had sessionid 0x13ca1e958bc0009
[junit4:junit4]   2> 10323 T499 oaz.ZooKeeper.close Session: 0x13ca1e958bc0009 closed
[junit4:junit4]   2> 10324 T499 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 10337 T499 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:43537/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@39cc1709
[junit4:junit4]   2> 10338 T499 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 10339 T569 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:43537. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 10339 T569 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:43537, initiating session
[junit4:junit4]   2> 10340 T501 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:19439
[junit4:junit4]   2> 10340 T501 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:19439
[junit4:junit4]   2> 10348 T503 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca1e958bc000a with negotiated timeout 20000 for client /140.211.11.196:19439
[junit4:junit4]   2> 10348 T569 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:43537, sessionid = 0x13ca1e958bc000a, negotiated timeout = 20000
[junit4:junit4]   2> 10348 T570 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@39cc1709 name:ZooKeeperConnection Watcher:127.0.0.1:43537/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 10349 T499 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 10350 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc000a type:create cxid:0x1 zxid:0x8d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 10351 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc000a type:create cxid:0x2 zxid:0x8e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 10353 T499 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48808_lt_
[junit4:junit4]   2> 10354 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc000a type:delete cxid:0x4 zxid:0x8f txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:48808_lt_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:48808_lt_
[junit4:junit4]   2> 10362 T499 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:48808_lt_
[junit4:junit4]   2> 10364 T521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10365 T527 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10365 T555 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 10365 T555 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 10366 T540 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 10366 T540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 10367 T521 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 10367 T527 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 10371 T499 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 10380 T571 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359926099918/collection1
[junit4:junit4]   2> 10380 T571 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 10381 T571 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 10381 T571 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 10383 T571 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359926099918/collection1/'
[junit4:junit4]   2> 10384 T571 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359926099918/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 10385 T571 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359926099918/collection1/lib/README' to classloader
[junit4:junit4]   2> 10440 T571 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 10526 T571 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 10627 T571 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 10634 T571 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 11264 T571 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 11289 T571 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 11293 T571 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 11312 T571 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 11317 T571 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 11322 T571 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 11323 T571 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 11325 T571 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359926099918/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3/
[junit4:junit4]   2> 11325 T571 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@36298619
[junit4:junit4]   2> 11326 T571 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 11327 T571 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3 forceNew:false
[junit4:junit4]   2> 11327 T571 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3
[junit4:junit4]   2> 11328 T571 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3/index/
[junit4:junit4]   2> 11328 T571 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 11329 T571 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3/index forceNew:false
[junit4:junit4]   2> 11335 T571 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@61c28a10; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 11336 T571 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 11337 T571 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3/index
[junit4:junit4]   2> 11337 T571 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 11340 T571 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 11340 T571 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 11340 T571 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 11341 T571 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 11341 T571 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 11342 T571 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 11342 T571 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 11343 T571 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 11343 T571 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 11344 T571 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 11345 T571 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 11345 T571 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 11346 T571 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 11346 T571 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 11346 T571 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 11347 T571 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 11348 T571 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11348 T571 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11349 T571 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11349 T571 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11350 T571 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11351 T571 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11351 T571 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11352 T571 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11353 T571 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 11353 T571 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 11366 T571 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 11371 T571 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 11372 T571 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 11375 T571 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3
[junit4:junit4]   2> 11376 T571 oass.SolrIndexSearcher.<init> Opening Searcher@5b2f54cd main
[junit4:junit4]   2> 11377 T571 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 11378 T571 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 11378 T571 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 11385 T571 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 11443 T572 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5b2f54cd main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 11450 T571 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 11450 T571 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 11496 T522 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11497 T522 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:39198_lt_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:39198/lt_"}
[junit4:junit4]   2> 11502 T522 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:48808_lt_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:48808/lt_"}
[junit4:junit4]   2> 11503 T522 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 11503 T522 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 11514 T570 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11514 T540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11514 T527 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11514 T555 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11514 T521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12453 T571 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 12453 T571 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:48808/lt_ collection:collection1 shard:shard1
[junit4:junit4]   2> 12483 T571 oasc.ZkController.register We are http://127.0.0.1:48808/lt_/collection1/ and leader is http://127.0.0.1:59558/lt_/collection1/
[junit4:junit4]   2> 12484 T571 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:48808/lt_
[junit4:junit4]   2> 12484 T571 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 12484 T571 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C67 name=collection1 org.apache.solr.core.SolrCore@6fbe4c22 url=http://127.0.0.1:48808/lt_/collection1 node=127.0.0.1:48808_lt_ C67_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:48808_lt_, base_url=http://127.0.0.1:48808/lt_}
[junit4:junit4]   2> 12491 T573 C67 P48808 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 12491 T571 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12492 T573 C67 P48808 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 12493 T573 C67 P48808 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 12493 T573 C67 P48808 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 12493 T499 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 12494 T499 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 12494 T499 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 12495 T573 C67 P48808 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 12691 T499 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 12694 T499 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:56235
[junit4:junit4]   2> 12695 T499 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 12695 T499 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 12696 T499 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359926102423
[junit4:junit4]   2> 12696 T499 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359926102423/solr.xml
[junit4:junit4]   2> 12696 T499 oasc.CoreContainer.<init> New CoreContainer 8126726
[junit4:junit4]   2> 12697 T499 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359926102423/'
[junit4:junit4]   2> 12697 T499 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359926102423/'
[junit4:junit4]   2> 12752 T499 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 12753 T499 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 12753 T499 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 12754 T499 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 12754 T499 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 12754 T499 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 12755 T499 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 12755 T499 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 12756 T499 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 12756 T499 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 12780 T499 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 12799 T499 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:43537/solr
[junit4:junit4]   2> 12800 T499 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 12800 T499 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:43537 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@6678bc7
[junit4:junit4]   2> 12801 T499 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 12802 T583 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:43537. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 12802 T583 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:43537, initiating session
[junit4:junit4]   2> 12803 T501 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:45364
[junit4:junit4]   2> 12803 T501 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:45364
[junit4:junit4]   2> 12805 T503 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca1e958bc000b with negotiated timeout 20000 for client /140.211.11.196:45364
[junit4:junit4]   2> 12805 T583 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:43537, sessionid = 0x13ca1e958bc000b, negotiated timeout = 20000
[junit4:junit4]   2> 12806 T584 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6678bc7 name:ZooKeeperConnection Watcher:127.0.0.1:43537 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 12806 T499 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 12807 T504 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13ca1e958bc000b
[junit4:junit4]   2> 12819 T584 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 12819 T501 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:45364 which had sessionid 0x13ca1e958bc000b
[junit4:junit4]   2> 12819 T499 oaz.ZooKeeper.close Session: 0x13ca1e958bc000b closed
[junit4:junit4]   2> 12820 T499 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 12834 T499 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:43537/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@48a02f43
[junit4:junit4]   2> 12835 T499 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 12835 T585 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:43537. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 12836 T585 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:43537, initiating session
[junit4:junit4]   2> 12836 T501 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:34033
[junit4:junit4]   2> 12836 T501 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:34033
[junit4:junit4]   2> 12838 T503 oazs.ZooKeeperServer.finishSessionInit Established session 0x13ca1e958bc000c with negotiated timeout 20000 for client /140.211.11.196:34033
[junit4:junit4]   2> 12838 T585 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:43537, sessionid = 0x13ca1e958bc000c, negotiated timeout = 20000
[junit4:junit4]   2> 12838 T586 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@48a02f43 name:ZooKeeperConnection Watcher:127.0.0.1:43537/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 12838 T499 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 12840 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc000c type:create cxid:0x1 zxid:0x9f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12841 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc000c type:create cxid:0x2 zxid:0xa0 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12843 T499 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56235_lt_
[junit4:junit4]   2> 12844 T504 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13ca1e958bc000c type:delete cxid:0x4 zxid:0xa1 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:56235_lt_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:56235_lt_
[junit4:junit4]   2> 12851 T499 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:56235_lt_
[junit4:junit4]   2> 12854 T570 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12854 T521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12854 T527 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12855 T555 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 12855 T540 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 12855 T555 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 12855 T540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 12857 T521 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 12858 T570 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 12858 T527 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 12860 T499 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 12867 T587 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359926102423/collection1
[junit4:junit4]   2> 12867 T587 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 12868 T587 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 12868 T587 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 12870 T587 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359926102423/collection1/'
[junit4:junit4]   2> 12871 T587 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359926102423/collection1/lib/README' to classloader
[junit4:junit4]   2> 12872 T587 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359926102423/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 12925 T587 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 12997 T587 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 13029 T522 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 13030 T522 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:48808_lt_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:48808/lt_"}
[junit4:junit4]   2> 13036 T570 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 13036 T521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 13036 T555 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 13036 T527 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 13036 T540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 13098 T587 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 13098 T586 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 13105 T587 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 13728 T587 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 13740 T587 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 13744 T587 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 13763 T587 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 13768 T587 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 13773 T587 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 13774 T587 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 13776 T587 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359926102423/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4/
[junit4:junit4]   2> 13776 T587 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@36298619
[junit4:junit4]   2> 13777 T587 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 13778 T587 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4 forceNew:false
[junit4:junit4]   2> 13779 T587 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4
[junit4:junit4]   2> 13779 T587 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4/index/
[junit4:junit4]   2> 13779 T587 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 13781 T587 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4/index forceNew:false
[junit4:junit4]   2> 13787 T587 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3df9fbf9; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 13788 T587 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 13789 T587 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4/index
[junit4:junit4]   2> 13789 T587 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 13791 T587 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 13792 T587 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 13792 T587 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 13792 T587 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 13793 T587 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 13793 T587 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 13794 T587 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 13794 T587 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 13795 T587 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 13796 T587 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 13796 T587 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 13797 T587 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 13798 T587 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 13798 T587 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 13798 T587 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 13799 T587 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 13799 T587 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13800 T587 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13801 T587 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13801 T587 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13802 T587 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13803 T587 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13803 T587 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13804 T587 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13804 T587 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 13805 T587 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 13818 T587 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 13823 T587 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 13825 T587 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 13827 T587 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4
[junit4:junit4]   2> 13829 T587 oass.SolrIndexSearcher.<init> Opening Searcher@7ec4e721 main
[junit4:junit4]   2> 13830 T587 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 13830 T587 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 13831 T587 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 13838 T587 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 13893 T588 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7ec4e721 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 13900 T587 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 13900 T587 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 14548 T522 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14549 T522 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:56235_lt_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:56235/lt_"}
[junit4:junit4]   2> 14550 T522 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 14550 T522 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 14563 T586 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14563 T527 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14563 T555 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14563 T540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14563 T570 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14563 T521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14903 T587 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 14903 T587 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:56235/lt_ collection:collection1 shard:shard2
[junit4:junit4]   2> 14907 T587 oasc.ZkController.register We are http://127.0.0.1:56235/lt_/collection1/ and leader is http://127.0.0.1:39198/lt_/collection1/
[junit4:junit4]   2> 14907 T587 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:56235/lt_
[junit4:junit4]   2> 14907 T587 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 14908 T587 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C68 name=collection1 org.apache.solr.core.SolrCore@56dc599f url=http://127.0.0.1:56235/lt_/collection1 node=127.0.0.1:56235_lt_ C68_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:56235_lt_, base_url=http://127.0.0.1:56235/lt_}
[junit4:junit4]   2> 14909 T589 C68 P56235 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 14909 T589 C68 P56235 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 14910 T587 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14910 T589 C68 P56235 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 14910 T589 C68 P56235 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 14911 T499 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 14911 T499 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 14912 T589 C68 P56235 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 14912 T499 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 14938 T499 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14940 T499 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4]   2> 14941 T499 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C67_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:48808_lt_, base_url=http://127.0.0.1:48808/lt_}
[junit4:junit4]   2> 15545 T573 C67 P48808 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:59558/lt_/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 15555 T573 C67 P48808 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 15576 T573 C67 P48808 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:48808/lt_ START replicas=[http://127.0.0.1:59558/lt_/collection1/] nUpdates=100
[junit4:junit4]   2> 15586 T573 C67 P48808 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 15587 T573 C67 P48808 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 15587 T573 C67 P48808 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 15588 T573 C67 P48808 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 15588 T573 C67 P48808 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 15588 T573 C67 P48808 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:59558/lt_/collection1/. core=collection1
[junit4:junit4]   2> 15588 T573 C67 P48808 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C69 name=collection1 org.apache.solr.core.SolrCore@7a561b47 url=http://127.0.0.1:59558/lt_/collection1 node=127.0.0.1:59558_lt_ C69_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:59558_lt_, base_url=http://127.0.0.1:59558/lt_, leader=true}
[junit4:junit4]   2> 15604 T531 C69 P59558 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 15611 T532 C69 P59558 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 15612 T532 C69 P59558 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty1
[junit4:junit4]   2> 15616 T532 C69 P59558 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7bf4515f; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 15617 T532 C69 P59558 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 15621 T532 C69 P59558 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7bf4515f; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7bf4515f; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 15621 T532 C69 P59558 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 15622 T532 C69 P59558 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty1
[junit4:junit4]   2> 15623 T532 C69 P59558 oass.SolrIndexSearcher.<init> Opening Searcher@65c40485 realtime
[junit4:junit4]   2> 15623 T532 C69 P59558 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15624 T532 C69 P59558 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 14
[junit4:junit4]   2> 15625 T573 C67 P48808 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 15625 T573 C67 P48808 oascsi.HttpClientUtil.createClient Creating new http client, config:connTimeout=5000&socketTimeout=20000&allowCompression=false&maxConnections=10000&maxConnectionsPerHost=10000
[junit4:junit4]   2> 15653 T573 C67 P48808 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 15656 T533 C69 P59558 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 15657 T533 C69 P59558 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 15658 T573 C67 P48808 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 15658 T573 C67 P48808 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 15658 T573 C67 P48808 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 15662 T533 C69 P59558 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty1
[junit4:junit4]   2> 15663 T533 C69 P59558 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty1/index
[junit4:junit4]   2> 15663 T533 C69 P59558 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=2 
[junit4:junit4]   2> 15664 T573 C67 P48808 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 15665 T573 C67 P48808 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3/index.20130203221505568 forceNew:false
[junit4:junit4]   2> 15666 T573 C67 P48808 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3
[junit4:junit4]   2> 15667 T573 C67 P48808 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3/index.20130203221505568 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2db7be4d; maxCacheMB=48.0 maxMergeSizeMB=4.0)) fullCopy=true
[junit4:junit4]   2> 15681 T533 C69 P59558 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=6 
[junit4:junit4]   2> 15688 T573 C67 P48808 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3/index
[junit4:junit4]   2> 15689 T573 C67 P48808 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 15689 T573 C67 P48808 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130203221505568
[junit4:junit4]   2> 15691 T573 C67 P48808 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3
[junit4:junit4]   2> 15692 T573 C67 P48808 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3
[junit4:junit4]   2> 15692 T573 C67 P48808 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3/index/ new=./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3/index.20130203221505568
[junit4:junit4]   2> 15696 T573 C67 P48808 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3/index.20130203221505568 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2db7be4d; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 15696 T573 C67 P48808 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 15698 T573 C67 P48808 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3
[junit4:junit4]   2> 15703 T573 C67 P48808 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 15703 T573 C67 P48808 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 15703 T573 C67 P48808 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 15704 T573 C67 P48808 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3/index.20130203221505568
[junit4:junit4]   2> 15705 T573 C67 P48808 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3
[junit4:junit4]   2> 15709 T573 C67 P48808 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3/index.20130203221505568 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2db7be4d; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 15709 T573 C67 P48808 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 15710 T573 C67 P48808 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 15710 T573 C67 P48808 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3
[junit4:junit4]   2> 15711 T573 C67 P48808 oass.SolrIndexSearcher.<init> Opening Searcher@3c2d6104 main
[junit4:junit4]   2> 15717 T572 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3c2d6104 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 15718 T572 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3/index
[junit4:junit4]   2> 15719 T573 C67 P48808 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 15776 T573 C67 P48808 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3/index.20130203221505568 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2db7be4d; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3/index.20130203221505568 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2db7be4d; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 15777 T573 C67 P48808 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 15778 T573 C67 P48808 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3
[junit4:junit4]   2> 15778 T573 C67 P48808 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15779 T573 C67 P48808 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3/index.20130203221505568
[junit4:junit4]   2> 15779 T573 C67 P48808 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3/index
[junit4:junit4]   2> 15779 T573 C67 P48808 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 15779 T573 C67 P48808 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 15780 T573 C67 P48808 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 15780 T573 C67 P48808 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 15806 T573 C67 P48808 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 15943 T499 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16069 T522 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16070 T522 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:56235_lt_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:56235/lt_"}
[junit4:junit4]   2> 16078 T522 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:48808_lt_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:48808/lt_"}
[junit4:junit4]   2> 16095 T570 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16095 T540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16095 T527 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16095 T555 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16095 T521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16095 T586 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16946 T499 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17950 T499 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C68_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:56235_lt_, base_url=http://127.0.0.1:56235/lt_}
[junit4:junit4]   2> 18942 T589 C68 P56235 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:39198/lt_/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 18942 T589 C68 P56235 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:56235/lt_ START replicas=[http://127.0.0.1:39198/lt_/collection1/] nUpdates=100
[junit4:junit4]   2> 18943 T589 C68 P56235 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 18943 T589 C68 P56235 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 18943 T589 C68 P56235 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 18943 T589 C68 P56235 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 18944 T589 C68 P56235 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 18945 T589 C68 P56235 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:39198/lt_/collection1/. core=collection1
[junit4:junit4]   2> 18945 T589 C68 P56235 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 18953 T499 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> ASYNC  NEW_CORE C70 name=collection1 org.apache.solr.core.SolrCore@b2471fb url=http://127.0.0.1:39198/lt_/collection1 node=127.0.0.1:39198_lt_ C70_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:39198_lt_, base_url=http://127.0.0.1:39198/lt_, leader=true}
[junit4:junit4]   2> 18955 T546 C70 P39198 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 18965 T547 C70 P39198 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 18966 T547 C70 P39198 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty2
[junit4:junit4]   2> 18970 T547 C70 P39198 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@254256b1; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 18971 T547 C70 P39198 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 18982 T547 C70 P39198 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@254256b1; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@254256b1; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 18982 T547 C70 P39198 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 18984 T547 C70 P39198 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty2
[junit4:junit4]   2> 18984 T547 C70 P39198 oass.SolrIndexSearcher.<init> Opening Searcher@4aef3675 realtime
[junit4:junit4]   2> 18985 T547 C70 P39198 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 18985 T547 C70 P39198 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 20
[junit4:junit4]   2> 18986 T589 C68 P56235 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 18986 T589 C68 P56235 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 18989 T548 C70 P39198 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 18989 T548 C70 P39198 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 18990 T589 C68 P56235 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 18991 T589 C68 P56235 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 18991 T589 C68 P56235 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 18994 T548 C70 P39198 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty2
[junit4:junit4]   2> 18994 T548 C70 P39198 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty2/index
[junit4:junit4]   2> 18995 T548 C70 P39198 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=2 
[junit4:junit4]   2> 18996 T589 C68 P56235 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 18997 T589 C68 P56235 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4/index.20130203221508900 forceNew:false
[junit4:junit4]   2> 18998 T589 C68 P56235 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4
[junit4:junit4]   2> 18999 T589 C68 P56235 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4/index.20130203221508900 lockFactory=org.apache.lucene.store.NativeFSLockFactory@f6d976e; maxCacheMB=48.0 maxMergeSizeMB=4.0)) fullCopy=true
[junit4:junit4]   2> 19002 T548 C70 P39198 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> 19004 T589 C68 P56235 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4/index
[junit4:junit4]   2> 19004 T589 C68 P56235 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 19005 T589 C68 P56235 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130203221508900
[junit4:junit4]   2> 19006 T589 C68 P56235 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4
[junit4:junit4]   2> 19007 T589 C68 P56235 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4
[junit4:junit4]   2> 19007 T589 C68 P56235 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4/index/ new=./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4/index.20130203221508900
[junit4:junit4]   2> 19011 T589 C68 P56235 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4/index.20130203221508900 lockFactory=org.apache.lucene.store.NativeFSLockFactory@f6d976e; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 19012 T589 C68 P56235 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 19014 T589 C68 P56235 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4
[junit4:junit4]   2> 19014 T589 C68 P56235 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 19014 T589 C68 P56235 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 19014 T589 C68 P56235 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 19015 T589 C68 P56235 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4/index.20130203221508900
[junit4:junit4]   2> 19016 T589 C68 P56235 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4
[junit4:junit4]   2> 19021 T589 C68 P56235 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4/index.20130203221508900 lockFactory=org.apache.lucene.store.NativeFSLockFactory@f6d976e; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 19021 T589 C68 P56235 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 19021 T589 C68 P56235 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 19022 T589 C68 P56235 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4
[junit4:junit4]   2> 19022 T589 C68 P56235 oass.SolrIndexSearcher.<init> Opening Searcher@692cbe60 main
[junit4:junit4]   2> 19024 T588 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@692cbe60 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 19025 T588 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4/index
[junit4:junit4]   2> 19025 T589 C68 P56235 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 19030 T589 C68 P56235 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4/index.20130203221508900 lockFactory=org.apache.lucene.store.NativeFSLockFactory@f6d976e; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4/index.20130203221508900 lockFactory=org.apache.lucene.store.NativeFSLockFactory@f6d976e; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 19030 T589 C68 P56235 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 19032 T589 C68 P56235 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4
[junit4:junit4]   2> 19032 T589 C68 P56235 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 19032 T589 C68 P56235 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4/index.20130203221508900
[junit4:junit4]   2> 19033 T589 C68 P56235 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4/index
[junit4:junit4]   2> 19033 T589 C68 P56235 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 19033 T589 C68 P56235 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 19033 T589 C68 P56235 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 19033 T589 C68 P56235 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 19035 T589 C68 P56235 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 19115 T522 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19116 T522 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:56235_lt_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:56235/lt_"}
[junit4:junit4]   2> 19130 T570 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19131 T527 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19130 T555 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19130 T521 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19130 T586 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19130 T540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19956 T499 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19957 T499 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4]   2> ASYNC  NEW_CORE C71 name=collection1 org.apache.solr.core.SolrCore@6a68c9a1 url=http://127.0.0.1:25591/lt_/collection1 node=127.0.0.1:25591_lt_ C71_STATE=coll:control_collection core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:25591_lt_, base_url=http://127.0.0.1:25591/lt_, leader=true}
[junit4:junit4]   2> 19971 T511 C71 P25591 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/control/data
[junit4:junit4]   2> 19975 T511 C71 P25591 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@708ce5ef; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 19976 T511 C71 P25591 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 19977 T511 C71 P25591 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/control/data
[junit4:junit4]   2> 19978 T511 C71 P25591 oass.SolrIndexSearcher.<init> Opening Searcher@159adcf5 realtime
[junit4:junit4]   2> 19979 T511 C71 P25591 /update {wt=javabin&version=2} {deleteByQuery=*:* (-1425985880586190848)} 0 10
[junit4:junit4]   2> 19995 T549 C70 P39198 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty2
[junit4:junit4]   2> 20000 T534 C69 P59558 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty1
[junit4:junit4]   2>  C68_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:56235_lt_, base_url=http://127.0.0.1:56235/lt_}
[junit4:junit4]   2> 20010 T576 C68 P56235 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty4
[junit4:junit4]   2> 20011 T576 C68 P56235 /update {update.distrib=FROMLEADER&_version_=-1425985880611356672&update.from=http://127.0.0.1:39198/lt_/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1425985880611356672)} 0 3
[junit4:junit4]   2>  C67_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:48808_lt_, base_url=http://127.0.0.1:48808/lt_}
[junit4:junit4]   2> 20014 T560 C67 P48808 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359926089911/jetty3
[junit4:junit4]   2> 20015 T560 C67 P48808 /update {update.distrib=FROMLEADER&_version_=-1425985880617648128&update.from=http://127.0.0.1:59558/lt_/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1425985880617648128)} 0 3
[junit4:junit4]   2> 20016 T534 C69 P59558 /update {update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1425985880617648128)} 0 18
[junit4:junit4]   2> 20017 T549 C70 P39198 /update {wt=javabin&version=2} {deleteByQuery=*:* (-1425985880611356672)} 0 25
[junit4:junit4]   2> 20027 T511 C71 P25591 /update {wt=javabin&version=2} {add=[1 (1425985880641765376)]} 0 5
[junit4:junit4]   2> 20054 T560 C67 P48808 /update {distrib.from=http://127.0.0.1:59558/lt_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1425985880659591168)]} 0 6
[junit4:junit4]   2> 20055 T534 C69 P59558 /update {distrib.from=http://127.0.0.1:39198/lt_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1425985880659591168)]} 0 16
[junit4:junit4]   2> 20056 T549 C70 P39198 /update {wt=javabin&version=2} {add=[1]} 0 26
[junit4:junit4]   2> 20062 T511 C71 P25591 /update {wt=javabin&version=2} {add=[2 (1425985880681611264)]} 0 2
[junit4:junit4]   2> 20079 T576 C68 P56235 /update {distrib.from=http://127.0.0.1:39198/lt_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1425985880689999872)]} 0 5
[junit4:junit4]   2> 20080 T550 C70 P39198 /update {distrib.from=http://127.0.0.1:59558/lt_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[2 (1425985880689999872)]} 0 13
[junit4:junit4]   2> 20081 T535 C69 P59558 /update {wt=javabin&version=2} {add=[2]} 0 16
[junit4:junit4]   2> 20087 T511 C71 P25591 /update {wt=javabin&version=2} {add=[3 (1425985880707825664)]} 0 3
[junit4:junit4]   2> 20101 T576 C68 P56235 /update {distrib.from=http://127.0.0.1:39198/lt_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[3 (1425985880717262848)]} 0 3
[junit4:junit4]   2> 20102 T550 C70 P39198 /update {distrib.from=http://127.0.0.1:59558/lt_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[3 (1425985880717262848)]} 0 9
[junit4:junit4]   2> 20103 T535 C69 P59558 /update {wt=javabin&version=2} {add=[3]} 0 13
[junit4:junit4]   2> 20109 T511 C71 P25591 /update {wt=javabin&version=2} {add=[4 (1425985880730894336)]} 0 3
[junit4:junit4]   2> 20124 T560 C67 P48808 /update {distrib.from=http://127.0.0.1:59558/lt_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[4 (1425985880740331520)]} 0 3
[junit4:junit4]   2> 20125 T534 C69 P59558 /update {distrib.from=http://127.0.0.1:39198/lt_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[4 (1425985880740331520)]} 0 10
[junit4:junit4]   2> 20126 T549 C70 P39198 /update {wt=javabin&version=2} {add=[4]} 0 14
[junit4:junit4]   2> 20132 T511 C71 P25591 /update {wt=javabin&version=2} {add=[5 (1425985880755011584)]} 0 3
[junit4:junit4]   2> 20146 T576 C68 P56235 /update {distrib.from=http://127.0.0.1:39198/lt_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[5 (1425985880764448768)]} 0 3
[junit4:junit4]   2> 20147 T550 C70 P39198 /update {distrib.from=http://127.0.0.1:59558/lt_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[5 (1425985880764448768)]} 0 9
[junit4:junit4]   2> 20148 T535 C69 P59558 /update {wt=javabin&version=2} {add=[5]} 0 12
[junit4:junit4]   2> 20154 T511 C71 P25591 /update {wt=javabin&version=2} {add=[6 (1425985880778080256)]} 0 3
[junit4:junit4]   2> 20168 T576 C68 P56235 /update {distrib.from=http://127.0.0.1:39198/lt_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[6 (1425985880787517440)]} 0 3
[junit4:junit4]   2> 20169 T550 C70 P39198 /update {distrib.from=http://127.0.0.1:59558/lt_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[6 (1425985880787517440)]} 0 9
[junit4:junit4]   2> 20170 T535 C69 P59558 /update {wt=javabin&version=2} {add=[6]} 0 13
[junit4:junit4]   2> 20176 T511 C71 P25591 /update {wt=javabin&version=2} {add=[7 (1425985880801148928)]} 0 3
[junit4:junit4]   2> 20189 T576 C68 P56235 /update {distrib.from=http://127.0.0.1:39198/lt_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[7 (1425985880807440384)]} 0 3
[junit4:junit4]   2> 20190 T549 C70 P39198 /update {wt=javabin&version=2} {add=[7 (1425985880807440384)]} 0 11
[junit4:junit4]   2> 20196 T511 C71 P25591 /update {wt=javabin&version=2} {add=[8 (1425985880822120448)]} 0 3
[junit4:junit4]   2> 20211 T560 C67 P48808 /update {distrib.from=http://127.0.0.1:59558/lt_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[8 (1425985880831557632)]} 0 3
[junit4:junit4]   2> 20212 T534 C69 P59558 /update {distrib.from=http://127.0.0.1:39198/lt_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[8 (1425985880831557632)]} 0 10
[junit4:junit4]   2> 20213 T549 C70 P39198 /update {wt=javabin&version=2} {add=[8]} 0 14
[junit4:junit4]   2> 20219 T511 C71 P25591 /update {wt=javabin&version=2} {add=[9 (1425985880846237696)]} 0 3
[junit4:junit4]   2> 20233 T576 C68 P56235 /update {distrib.from=http://127.0.0.1:39198/lt_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[9 (1425985880855674880)]} 0 3
[junit4:junit4]   2> 20233 T550 C70 P39198 /update {distrib.from=http://127.0.0.1:59558/lt_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[9 (1425985880855674880)]} 0 8
[junit4:junit4]   2> 20234 T535 C69 P59558 /update {wt=javabin&version=2} {add=[9]} 0 12
[junit4:junit4]   2> 20261 T511 C71 P25591 /update {wt=javabin&version=2} {add=[10 (1425985880869306368)]} 0 23
[junit4:junit4]   2> 20276 T560 C67 P48808 /update {distrib.from=http://127.0.0.1:59558/lt_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10 (1425985880900763648)]} 0 3
[junit4:junit4]   2> 20276 T534 C69 P59558 /update {distrib.from=http://127.0.0.1:39198/lt_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[10 (1425985880900763648)]} 0 8
[junit4:junit4]   2> 20277 T549 C70 P39198 /update {wt=javabin&version=2} {add=[10]} 0 12
[junit4:junit4]   2> 20284 T511 C71 P25591 /update {wt=javabin&version=2} {add=[11 (1425985880913346560)]} 0 4
[junit4:junit4]   2> 20298 T560 C67 P48808 /update {distrib.from=http://127.0.0.1:59558/lt_/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[11 (1425985880923832320)]} 0 3
[junit4:junit4]   2> 20299 T534 C69 P59558 /update {distrib.from=http://127.0.0.1:39198/lt_/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[11 (1425985880923832320)]} 0 9
[junit4:junit4]   2> 20299 T549 C70 P39198 /update {wt=javabin&version=2} {add=[11]} 0 12
[junit4:junit4]   2> 20306 T511 C71 P25591 /update {wt=javabin&version=2} {add=[12 (1425985880937463808)]} 0 3
[junit4:junit4]   2> 20320 T560 C67 P48808 /update {distrib.from=http://127.0.0.1:59558/lt_/collection1/&update.distrib=FROMLEADER&wt=

[...truncated too long message...]

rdownChained.java:50)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4:junit4]   2> 	        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]   2> 	        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  22) Thread[id=572, name=searcherExecutor-469-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  23) Thread[id=629, name=searcherExecutor-496-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> NOTE: test params are: codec=Lucene42: {n_f1=PostingsFormat(name=SimpleText), foo_b=PostingsFormat(name=MockSep), foo_d=MockVariableIntBlock(baseBlockSize=77), foo_f=PostingsFormat(name=MockSep), n_tl1=PostingsFormat(name=SimpleText), n_d1=MockFixedIntBlock(blockSize=1003), rnd_b=MockVariableIntBlock(baseBlockSize=77), intDefault=MockFixedIntBlock(blockSize=1003), n_td1=PostingsFormat(name=SimpleText), timestamp=MockFixedIntBlock(blockSize=1003), id=MockVariableIntBlock(baseBlockSize=77), range_facet_sl=PostingsFormat(name=MockSep), range_facet_si=PostingsFormat(name=SimpleText), oddField_s=PostingsFormat(name=MockSep), foo_i=PostingsFormat(name=SimpleText), multiDefault=PostingsFormat(name=SimpleText), n_tf1=MockFixedIntBlock(blockSize=1003), n_dt1=PostingsFormat(name=SimpleText), n_ti1=MockVariableIntBlock(baseBlockSize=77), range_facet_l=MockFixedIntBlock(blockSize=1003), text=PostingsFormat(name=MockSep), _version_=MockFixedIntBlock(blockSize=1003), val_i=MockVariableIntBlock(baseBlockSize=77), SubjectTerms_mfacet=MockFixedIntBlock(blockSize=1003), a_t=MockFixedIntBlock(blockSize=1003), n_tdt1=MockFixedIntBlock(blockSize=1003), other_tl1=MockFixedIntBlock(blockSize=1003), n_l1=MockFixedIntBlock(blockSize=1003), a_si=PostingsFormat(name=SimpleText)}, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=ro, timezone=Europe/Malta
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=25,free=255086936,total=470810624
[junit4:junit4]   2> NOTE: All tests run in this JVM: [QueryElevationComponentTest, SystemInfoHandlerTest, JSONWriterTest, CopyFieldTest, DateFieldTest, TestSuggestSpellingConverter, TestSearchPerf, DirectSolrConnectionTest, NumericFieldsTest, TestReload, SOLR749Test, IndexBasedSpellCheckerTest, SolrCmdDistributorTest, TestFuzzyAnalyzedSuggestions, FastVectorHighlighterTest, PreAnalyzedFieldTest, TestRangeQuery, TestComponentsName, LeaderElectionIntegrationTest, TestDefaultSimilarityFactory, TestPerFieldSimilarity, DirectUpdateHandlerOptimizeTest, NoCacheHeaderTest, TermsComponentTest, DefaultValueUpdateProcessorTest, TestSolrIndexConfig, TestPHPSerializedResponseWriter, SpellCheckComponentTest, TestBinaryResponseWriter, TestIndexingPerformance, MoreLikeThisHandlerTest, FieldAnalysisRequestHandlerTest, TestStressVersions, AlternateDirectoryTest, TestSolr4Spatial, UpdateRequestProcessorFactoryTest, TestPseudoReturnFields, AutoCommitTest, TestNumberUtils, TestFastOutputStream, BasicDistributedZkTest]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=BasicDistributedZkTest -Dtests.seed=437C1ACFB7351B0A -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ro -Dtests.timezone=Europe/Malta -Dtests.file.encoding=US-ASCII
[junit4:junit4] ERROR   0.00s J0 | BasicDistributedZkTest (suite) <<<
[junit4:junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([437C1ACFB7351B0A]:0)
[junit4:junit4] Completed on J0 in 7223.60s, 1 test, 2 errors <<< FAILURES!

[...truncated 74 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:346: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/common-build.xml:428: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:1176: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:841: There were test failures: 249 suites, 1038 tests, 1 suite-level error, 1 error, 66 ignored (5 assumptions)

Total time: 144 minutes 57 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



Re: [JENKINS] Lucene-Solr-Tests-trunk-java7 - Build # 3704 - Still Failing

Posted by Mark Miller <ma...@gmail.com>.
Made a change to try and address this. Looks like an http call from SnapPuller hanging on the black hole for dear life.

- Mark

On Feb 3, 2013, at 6:07 AM, Apache Jenkins Server <je...@builds.apache.org> wrote:

> Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-java7/3704/
> 
> 2 tests failed.
> FAILED:  org.apache.solr.cloud.BasicDistributedZkTest.testDistribSearch
> 
> Error Message:
> Test abandoned because suite timeout was reached.
> 
> Stack Trace:
> java.lang.Exception: Test abandoned because suite timeout was reached.
> 	at __randomizedtesting.SeedInfo.seed([B1AAA651BCD0851A]:0)
> 
> 
> FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest
> 
> Error Message:
> Suite timeout exceeded (>= 7200000 msec).
> 
> Stack Trace:
> java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
> 	at __randomizedtesting.SeedInfo.seed([B1AAA651BCD0851A]:0)
> 
> 
> 
> 
> Build Log:
> [...truncated 9416 lines...]
> [junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
> [junit4:junit4]   2> 9981 T3 oasc.CoreContainer.finalize SEVERE CoreContainer was not shutdown prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!  instance=1207232956
> [junit4:junit4]   2> 0 T429 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
> [junit4:junit4]   2> 5 T429 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
> [junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZkTest-1359882417336
> [junit4:junit4]   2> 6 T429 oasc.ZkTestServer.run STARTING ZK TEST SERVER
> [junit4:junit4]   2> 7 T430 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
> [junit4:junit4]   2> 10 T430 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
> [junit4:junit4]   2> 10 T430 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
> [junit4:junit4]   2> 10 T430 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
> [junit4:junit4]   2> 11 T430 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
> [junit4:junit4]   2> 12 T430 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/zookeeper/server1/data/version-2/snapshot.0
> [junit4:junit4]   2> 108 T429 oasc.ZkTestServer.run start zk server on port:18580
> [junit4:junit4]   2> 108 T429 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:18580 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@36729c7c
> [junit4:junit4]   2> 109 T429 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4]   2> 110 T435 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:18580. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4]   2> 110 T435 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:18580, initiating session
> [junit4:junit4]   2> 111 T431 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:55124
> [junit4:junit4]   2> 111 T431 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:55124
> [junit4:junit4]   2> 111 T433 oazsp.FileTxnLog.append Creating new log file: log.1
> [junit4:junit4]   2> 113 T433 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9f4ef4bf0000 with negotiated timeout 10000 for client /140.211.11.196:55124
> [junit4:junit4]   2> 114 T435 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:18580, sessionid = 0x13c9f4ef4bf0000, negotiated timeout = 10000
> [junit4:junit4]   2> 114 T436 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@36729c7c name:ZooKeeperConnection Watcher:127.0.0.1:18580 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4]   2> 114 T429 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4]   2> 115 T429 oascc.SolrZkClient.makePath makePath: /solr
> [junit4:junit4]   2> 118 T434 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c9f4ef4bf0000
> [junit4:junit4]   2> 132 T436 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4]   2> 132 T429 oaz.ZooKeeper.close Session: 0x13c9f4ef4bf0000 closed
> [junit4:junit4]   2> 132 T431 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:55124 which had sessionid 0x13c9f4ef4bf0000
> [junit4:junit4]   2> 132 T429 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:18580/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@50242f7d
> [junit4:junit4]   2> 133 T429 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4]   2> 133 T437 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:18580. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4]   2> 134 T437 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:18580, initiating session
> [junit4:junit4]   2> 134 T431 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:17574
> [junit4:junit4]   2> 134 T431 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:17574
> [junit4:junit4]   2> 136 T433 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9f4ef4bf0001 with negotiated timeout 10000 for client /140.211.11.196:17574
> [junit4:junit4]   2> 136 T437 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:18580, sessionid = 0x13c9f4ef4bf0001, negotiated timeout = 10000
> [junit4:junit4]   2> 136 T438 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@50242f7d name:ZooKeeperConnection Watcher:127.0.0.1:18580/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4]   2> 136 T429 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4]   2> 137 T429 oascc.SolrZkClient.makePath makePath: /collections/collection1
> [junit4:junit4]   2> 146 T429 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
> [junit4:junit4]   2> 150 T429 oascc.SolrZkClient.makePath makePath: /collections/control_collection
> [junit4:junit4]   2> 153 T429 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
> [junit4:junit4]   2> 161 T429 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
> [junit4:junit4]   2> 162 T429 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
> [junit4:junit4]   2> 176 T429 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
> [junit4:junit4]   2> 177 T429 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
> [junit4:junit4]   2> 282 T429 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
> [junit4:junit4]   2> 283 T429 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
> [junit4:junit4]   2> 286 T429 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
> [junit4:junit4]   2> 287 T429 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
> [junit4:junit4]   2> 297 T429 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
> [junit4:junit4]   2> 298 T429 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
> [junit4:junit4]   2> 301 T429 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
> [junit4:junit4]   2> 302 T429 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
> [junit4:junit4]   2> 306 T429 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
> [junit4:junit4]   2> 307 T429 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
> [junit4:junit4]   2> 311 T429 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
> [junit4:junit4]   2> 312 T429 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
> [junit4:junit4]   2> 315 T429 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
> [junit4:junit4]   2> 316 T429 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
> [junit4:junit4]   2> 319 T434 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c9f4ef4bf0001
> [junit4:junit4]   2> 320 T438 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4]   2> 320 T431 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:17574 which had sessionid 0x13c9f4ef4bf0001
> [junit4:junit4]   2> 320 T429 oaz.ZooKeeper.close Session: 0x13c9f4ef4bf0001 closed
> [junit4:junit4]   2> 486 T429 oejs.Server.doStart jetty-8.1.8.v20121106
> [junit4:junit4]   2> 490 T429 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:38869
> [junit4:junit4]   2> 491 T429 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4:junit4]   2> 491 T429 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4:junit4]   2> 492 T429 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359882417651
> [junit4:junit4]   2> 492 T429 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359882417651/solr.xml
> [junit4:junit4]   2> 493 T429 oasc.CoreContainer.<init> New CoreContainer 821516233
> [junit4:junit4]   2> 494 T429 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359882417651/'
> [junit4:junit4]   2> 494 T429 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359882417651/'
> [junit4:junit4]   2> 548 T429 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
> [junit4:junit4]   2> 548 T429 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4:junit4]   2> 549 T429 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4:junit4]   2> 549 T429 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4:junit4]   2> 550 T429 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4:junit4]   2> 550 T429 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4:junit4]   2> 551 T429 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4:junit4]   2> 551 T429 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4:junit4]   2> 551 T429 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4:junit4]   2> 552 T429 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
> [junit4:junit4]   2> 575 T429 oasc.CoreContainer.load Registering Log Listener
> [junit4:junit4]   2> 594 T429 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:18580/solr
> [junit4:junit4]   2> 595 T429 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4:junit4]   2> 595 T429 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:18580 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@15f502e3
> [junit4:junit4]   2> 596 T429 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4]   2> 597 T448 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:18580. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4]   2> 597 T448 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:18580, initiating session
> [junit4:junit4]   2> 598 T431 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:46186
> [junit4:junit4]   2> 598 T431 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:46186
> [junit4:junit4]   2> 600 T433 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9f4ef4bf0002 with negotiated timeout 20000 for client /140.211.11.196:46186
> [junit4:junit4]   2> 600 T448 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:18580, sessionid = 0x13c9f4ef4bf0002, negotiated timeout = 20000
> [junit4:junit4]   2> 600 T449 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@15f502e3 name:ZooKeeperConnection Watcher:127.0.0.1:18580 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4]   2> 601 T429 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4]   2> 602 T434 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c9f4ef4bf0002
> [junit4:junit4]   2> 603 T449 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4]   2> 603 T431 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:46186 which had sessionid 0x13c9f4ef4bf0002
> [junit4:junit4]   2> 603 T429 oaz.ZooKeeper.close Session: 0x13c9f4ef4bf0002 closed
> [junit4:junit4]   2> 604 T429 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
> [junit4:junit4]   2> 617 T429 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:18580/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@8682e3
> [junit4:junit4]   2> 618 T429 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4]   2> 618 T450 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:18580. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4]   2> 619 T450 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:18580, initiating session
> [junit4:junit4]   2> 619 T431 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:23653
> [junit4:junit4]   2> 619 T431 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:23653
> [junit4:junit4]   2> 621 T433 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9f4ef4bf0003 with negotiated timeout 20000 for client /140.211.11.196:23653
> [junit4:junit4]   2> 621 T450 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:18580, sessionid = 0x13c9f4ef4bf0003, negotiated timeout = 20000
> [junit4:junit4]   2> 621 T451 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@8682e3 name:ZooKeeperConnection Watcher:127.0.0.1:18580/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4]   2> 621 T429 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4]   2> 624 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 632 T429 oascc.SolrZkClient.makePath makePath: /live_nodes
> [junit4:junit4]   2> 635 T429 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38869_
> [junit4:junit4]   2> 635 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0003 type:delete cxid:0x7 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:38869_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:38869_
> [junit4:junit4]   2> 636 T429 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:38869_
> [junit4:junit4]   2> 646 T429 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
> [junit4:junit4]   2> 659 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0003 type:delete cxid:0x15 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
> [junit4:junit4]   2> 660 T429 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
> [junit4:junit4]   2> 663 T429 oasc.Overseer.start Overseer (id=89121254102990851-127.0.0.1:38869_-n_0000000000) starting
> [junit4:junit4]   2> 664 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0003 type:create cxid:0x1a zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 671 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0003 type:create cxid:0x1b zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 672 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0003 type:create cxid:0x1c zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 679 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0003 type:create cxid:0x1d zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 680 T453 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
> [junit4:junit4]   2> 681 T429 oascc.SolrZkClient.makePath makePath: /clusterstate.json
> [junit4:junit4]   2> 691 T429 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
> [junit4:junit4]   2> 694 T452 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
> [junit4:junit4]   2> 699 T454 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359882417651/collection1
> [junit4:junit4]   2> 699 T454 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
> [junit4:junit4]   2> 700 T454 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4:junit4]   2> 700 T454 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
> [junit4:junit4]   2> 702 T454 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359882417651/collection1/'
> [junit4:junit4]   2> 703 T454 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359882417651/collection1/lib/README' to classloader
> [junit4:junit4]   2> 703 T454 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359882417651/collection1/lib/classes/' to classloader
> [junit4:junit4]   2> 755 T454 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4:junit4]   2> 826 T454 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4:junit4]   2> 927 T454 oass.IndexSchema.readSchema Reading Solr Schema
> [junit4:junit4]   2> 935 T454 oass.IndexSchema.readSchema Schema name=test
> [junit4:junit4]   2> 1558 T454 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4:junit4]   2> 1566 T454 oass.IndexSchema.readSchema default search field in schema is text
> [junit4:junit4]   2> 1569 T454 oass.IndexSchema.readSchema unique key field: id
> [junit4:junit4]   2> 1583 T454 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4]   2> 1587 T454 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4]   2> 1590 T454 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4]   2> 1592 T454 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4]   2> 1593 T454 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359882417651/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/control/data/
> [junit4:junit4]   2> 1593 T454 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2bd1127
> [junit4:junit4]   2> 1593 T454 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
> [junit4:junit4]   2> 1594 T454 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/control/data forceNew:false
> [junit4:junit4]   2> 1595 T454 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/control/data
> [junit4:junit4]   2> 1595 T454 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/control/data/index/
> [junit4:junit4]   2> 1595 T454 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/control/data/index' doesn't exist. Creating new index...
> [junit4:junit4]   2> 1596 T454 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/control/data/index forceNew:false
> [junit4:junit4]   2> 1600 T454 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2385857),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4]   2> 1601 T454 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
> [junit4:junit4]   2> 1601 T454 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/control/data/index
> [junit4:junit4]   2> 1601 T454 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
> [junit4:junit4]   2> 1603 T454 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4:junit4]   2> 1603 T454 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
> [junit4:junit4]   2> 1603 T454 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4:junit4]   2> 1603 T454 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
> [junit4:junit4]   2> 1604 T454 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4:junit4]   2> 1604 T454 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4:junit4]   2> 1604 T454 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4:junit4]   2> 1605 T454 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4:junit4]   2> 1605 T454 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4:junit4]   2> 1605 T454 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
> [junit4:junit4]   2> 1606 T454 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
> [junit4:junit4]   2> 1606 T454 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4:junit4]   2> 1606 T454 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
> [junit4:junit4]   2> 1606 T454 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
> [junit4:junit4]   2> 1607 T454 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
> [junit4:junit4]   2> 1607 T454 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4:junit4]   2> 1607 T454 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 1608 T454 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 1608 T454 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 1608 T454 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 1608 T454 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 1609 T454 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 1609 T454 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 1610 T454 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 1610 T454 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
> [junit4:junit4]   2> 1610 T454 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
> [junit4:junit4]   2> 1618 T454 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4:junit4]   2> 1621 T454 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
> [junit4:junit4]   2> 1622 T454 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
> [junit4:junit4]   2> 1624 T454 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/control/data
> [junit4:junit4]   2> 1625 T454 oass.SolrIndexSearcher.<init> Opening Searcher@a3807eb main
> [junit4:junit4]   2> 1626 T454 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4:junit4]   2> 1626 T454 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4:junit4]   2> 1627 T454 oashc.SpellCheckComponent.inform Initializing spell checkers
> [junit4:junit4]   2> 1643 T454 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
> [junit4:junit4]   2> 1697 T455 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@a3807eb main{StandardDirectoryReader(segments_1:1)}
> [junit4:junit4]   2> 1700 T454 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4]   2> 1700 T454 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4]   2> 1701 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0003 type:create cxid:0x50 zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
> [junit4:junit4]   2> 2198 T452 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 2199 T452 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
> [junit4:junit4]   2> 	  "operation":"state",
> [junit4:junit4]   2> 	  "numShards":"1",
> [junit4:junit4]   2> 	  "shard":null,
> [junit4:junit4]   2> 	  "roles":null,
> [junit4:junit4]   2> 	  "state":"down",
> [junit4:junit4]   2> 	  "core":"collection1",
> [junit4:junit4]   2> 	  "collection":"control_collection",
> [junit4:junit4]   2> 	  "node_name":"127.0.0.1:38869_",
> [junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:38869"}
> [junit4:junit4]   2> 2199 T452 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
> [junit4:junit4]   2> 2204 T452 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
> [junit4:junit4]   2> 2205 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0003 type:create cxid:0x58 zxid:0x2c txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
> [junit4:junit4]   2> 2227 T451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4:junit4]   2> 2713 T454 oasc.CoreContainer.registerCore registering core: collection1
> [junit4:junit4]   2> 2713 T454 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:38869 collection:control_collection shard:shard1
> [junit4:junit4]   2> 2714 T454 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
> [junit4:junit4]   2> 2732 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0003 type:delete cxid:0x72 zxid:0x36 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
> [junit4:junit4]   2> 2733 T454 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
> [junit4:junit4]   2> 2734 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0003 type:create cxid:0x73 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 2736 T454 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
> [junit4:junit4]   2> 2736 T454 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
> [junit4:junit4]   2> 2736 T454 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:38869/collection1/
> [junit4:junit4]   2> 2737 T454 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
> [junit4:junit4]   2> 2737 T454 oasc.SyncStrategy.syncToMe http://127.0.0.1:38869/collection1/ has no replicas
> [junit4:junit4]   2> 2737 T454 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:38869/collection1/
> [junit4:junit4]   2> 2738 T454 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
> [junit4:junit4]   2> 2742 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0003 type:create cxid:0x7d zxid:0x3b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 3733 T452 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 3756 T451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
> [junit4:junit4]   2> 3765 T454 oasc.ZkController.register We are http://127.0.0.1:38869/collection1/ and leader is http://127.0.0.1:38869/collection1/
> [junit4:junit4]   2> 3765 T454 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:38869
> [junit4:junit4]   2> 3765 T454 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
> [junit4:junit4]   2> 3765 T454 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4:junit4]   2> 3765 T454 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4]   2> 3768 T454 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 3769 T429 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
> [junit4:junit4]   2> 3769 T429 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4:junit4]   2> 3770 T429 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4]   2> 3784 T429 oascsi.HttpClientUtil.createClient Creating new http client, config:
> [junit4:junit4]   2> 3794 T429 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:18580/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@4ecdac83
> [junit4:junit4]   2> 3795 T429 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4]   2> 3795 T456 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:18580. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4]   2> 3796 T456 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:18580, initiating session
> [junit4:junit4]   2> 3796 T431 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:48743
> [junit4:junit4]   2> 3796 T431 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:48743
> [junit4:junit4]   2> 3799 T433 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9f4ef4bf0004 with negotiated timeout 10000 for client /140.211.11.196:48743
> [junit4:junit4]   2> 3799 T456 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:18580, sessionid = 0x13c9f4ef4bf0004, negotiated timeout = 10000
> [junit4:junit4]   2> 3799 T457 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4ecdac83 name:ZooKeeperConnection Watcher:127.0.0.1:18580/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4]   2> 3799 T429 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4]   2> 3800 T429 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
> [junit4:junit4]   2> 3804 T429 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
> [junit4:junit4]   2> 3935 T429 oejs.Server.doStart jetty-8.1.8.v20121106
> [junit4:junit4]   2> 3938 T429 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:55544
> [junit4:junit4]   2> 3939 T429 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4:junit4]   2> 3939 T429 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4:junit4]   2> 3939 T429 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359882421134
> [junit4:junit4]   2> 3940 T429 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359882421134/solr.xml
> [junit4:junit4]   2> 3940 T429 oasc.CoreContainer.<init> New CoreContainer 123419126
> [junit4:junit4]   2> 3940 T429 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359882421134/'
> [junit4:junit4]   2> 3941 T429 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359882421134/'
> [junit4:junit4]   2> 3976 T429 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
> [junit4:junit4]   2> 3976 T429 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4:junit4]   2> 3977 T429 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4:junit4]   2> 3977 T429 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4:junit4]   2> 3977 T429 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4:junit4]   2> 3978 T429 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4:junit4]   2> 3978 T429 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4:junit4]   2> 3978 T429 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4:junit4]   2> 3978 T429 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4:junit4]   2> 3979 T429 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
> [junit4:junit4]   2> 3993 T429 oasc.CoreContainer.load Registering Log Listener
> [junit4:junit4]   2> 4007 T429 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:18580/solr
> [junit4:junit4]   2> 4007 T429 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4:junit4]   2> 4007 T429 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:18580 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@34fccead
> [junit4:junit4]   2> 4008 T429 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4]   2> 4009 T467 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:18580. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4]   2> 4009 T467 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:18580, initiating session
> [junit4:junit4]   2> 4009 T431 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:49421
> [junit4:junit4]   2> 4010 T431 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:49421
> [junit4:junit4]   2> 4011 T433 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9f4ef4bf0005 with negotiated timeout 20000 for client /140.211.11.196:49421
> [junit4:junit4]   2> 4011 T467 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:18580, sessionid = 0x13c9f4ef4bf0005, negotiated timeout = 20000
> [junit4:junit4]   2> 4011 T468 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@34fccead name:ZooKeeperConnection Watcher:127.0.0.1:18580 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4]   2> 4011 T429 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4]   2> 4012 T434 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c9f4ef4bf0005
> [junit4:junit4]   2> 4014 T431 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:49421 which had sessionid 0x13c9f4ef4bf0005
> [junit4:junit4]   2> 4014 T468 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4]   2> 4013 T429 oaz.ZooKeeper.close Session: 0x13c9f4ef4bf0005 closed
> [junit4:junit4]   2> 4014 T429 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
> [junit4:junit4]   2> 4022 T429 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:18580/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@72cd6a83
> [junit4:junit4]   2> 4023 T429 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4]   2> 4023 T469 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:18580. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4]   2> 4023 T469 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:18580, initiating session
> [junit4:junit4]   2> 4023 T431 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:43721
> [junit4:junit4]   2> 4024 T431 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:43721
> [junit4:junit4]   2> 4025 T433 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9f4ef4bf0006 with negotiated timeout 20000 for client /140.211.11.196:43721
> [junit4:junit4]   2> 4025 T469 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:18580, sessionid = 0x13c9f4ef4bf0006, negotiated timeout = 20000
> [junit4:junit4]   2> 4025 T470 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@72cd6a83 name:ZooKeeperConnection Watcher:127.0.0.1:18580/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4]   2> 4025 T429 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4]   2> 4026 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0006 type:create cxid:0x1 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 4027 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0006 type:create cxid:0x2 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 4036 T429 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55544_
> [junit4:junit4]   2> 4037 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0006 type:delete cxid:0x4 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:55544_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:55544_
> [junit4:junit4]   2> 4038 T429 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:55544_
> [junit4:junit4]   2> 4040 T451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
> [junit4:junit4]   2> 4041 T457 oascc.ZkStateReader$3.process Updating live nodes... (2)
> [junit4:junit4]   2> 4042 T451 oascc.ZkStateReader$3.process Updating live nodes... (2)
> [junit4:junit4]   2> 4049 T429 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
> [junit4:junit4]   2> 4055 T471 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359882421134/collection1
> [junit4:junit4]   2> 4056 T471 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
> [junit4:junit4]   2> 4056 T471 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4:junit4]   2> 4056 T471 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
> [junit4:junit4]   2> 4058 T471 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359882421134/collection1/'
> [junit4:junit4]   2> 4059 T471 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359882421134/collection1/lib/classes/' to classloader
> [junit4:junit4]   2> 4059 T471 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359882421134/collection1/lib/README' to classloader
> [junit4:junit4]   2> 4099 T471 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4:junit4]   2> 4151 T471 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4:junit4]   2> 4252 T471 oass.IndexSchema.readSchema Reading Solr Schema
> [junit4:junit4]   2> 4258 T471 oass.IndexSchema.readSchema Schema name=test
> [junit4:junit4]   2> 4808 T471 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4:junit4]   2> 4821 T471 oass.IndexSchema.readSchema default search field in schema is text
> [junit4:junit4]   2> 4825 T471 oass.IndexSchema.readSchema unique key field: id
> [junit4:junit4]   2> 4842 T471 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4]   2> 4848 T471 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4]   2> 4853 T471 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4]   2> 4855 T471 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4]   2> 4857 T471 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359882421134/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty1/
> [junit4:junit4]   2> 4857 T471 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2bd1127
> [junit4:junit4]   2> 4858 T471 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
> [junit4:junit4]   2> 4858 T471 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty1 forceNew:false
> [junit4:junit4]   2> 4859 T471 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty1
> [junit4:junit4]   2> 4859 T471 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty1/index/
> [junit4:junit4]   2> 4860 T471 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty1/index' doesn't exist. Creating new index...
> [junit4:junit4]   2> 4861 T471 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty1/index forceNew:false
> [junit4:junit4]   2> 4868 T471 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@45782220),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4]   2> 4868 T471 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
> [junit4:junit4]   2> 4869 T471 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty1/index
> [junit4:junit4]   2> 4869 T471 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
> [junit4:junit4]   2> 4872 T471 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4:junit4]   2> 4872 T471 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
> [junit4:junit4]   2> 4872 T471 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4:junit4]   2> 4873 T471 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
> [junit4:junit4]   2> 4873 T471 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4:junit4]   2> 4874 T471 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4:junit4]   2> 4874 T471 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4:junit4]   2> 4875 T471 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4:junit4]   2> 4875 T471 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4:junit4]   2> 4876 T471 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
> [junit4:junit4]   2> 4876 T471 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
> [junit4:junit4]   2> 4877 T471 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4:junit4]   2> 4877 T471 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
> [junit4:junit4]   2> 4878 T471 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
> [junit4:junit4]   2> 4878 T471 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
> [junit4:junit4]   2> 4879 T471 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4:junit4]   2> 4879 T471 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 4880 T471 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 4880 T471 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 4881 T471 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 4881 T471 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 4882 T471 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 4883 T471 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 4883 T471 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 4884 T471 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
> [junit4:junit4]   2> 4884 T471 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
> [junit4:junit4]   2> 4897 T471 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4:junit4]   2> 4902 T471 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
> [junit4:junit4]   2> 4903 T471 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
> [junit4:junit4]   2> 4906 T471 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty1
> [junit4:junit4]   2> 4908 T471 oass.SolrIndexSearcher.<init> Opening Searcher@3b6be770 main
> [junit4:junit4]   2> 4909 T471 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4:junit4]   2> 4909 T471 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4:junit4]   2> 4909 T471 oashc.SpellCheckComponent.inform Initializing spell checkers
> [junit4:junit4]   2> 4917 T471 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
> [junit4:junit4]   2> 4983 T472 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3b6be770 main{StandardDirectoryReader(segments_1:1)}
> [junit4:junit4]   2> 4989 T471 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4]   2> 4989 T471 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4]   2> 5264 T452 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 5265 T452 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
> [junit4:junit4]   2> 	  "operation":"state",
> [junit4:junit4]   2> 	  "numShards":"1",
> [junit4:junit4]   2> 	  "shard":"shard1",
> [junit4:junit4]   2> 	  "roles":null,
> [junit4:junit4]   2> 	  "state":"active",
> [junit4:junit4]   2> 	  "core":"collection1",
> [junit4:junit4]   2> 	  "collection":"control_collection",
> [junit4:junit4]   2> 	  "node_name":"127.0.0.1:38869_",
> [junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:38869"}
> [junit4:junit4]   2> 5272 T452 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4]   2> 	  "operation":"state",
> [junit4:junit4]   2> 	  "numShards":"2",
> [junit4:junit4]   2> 	  "shard":null,
> [junit4:junit4]   2> 	  "roles":null,
> [junit4:junit4]   2> 	  "state":"down",
> [junit4:junit4]   2> 	  "core":"collection1",
> [junit4:junit4]   2> 	  "collection":"collection1",
> [junit4:junit4]   2> 	  "node_name":"127.0.0.1:55544_",
> [junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:55544"}
> [junit4:junit4]   2> 5272 T452 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
> [junit4:junit4]   2> 5273 T452 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
> [junit4:junit4]   2> 5287 T451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4]   2> 5287 T470 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4]   2> 5288 T457 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4]   2> 5992 T471 oasc.CoreContainer.registerCore registering core: collection1
> [junit4:junit4]   2> 5992 T471 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:55544 collection:collection1 shard:shard1
> [junit4:junit4]   2> 5993 T471 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
> [junit4:junit4]   2> 6037 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0006 type:delete cxid:0x43 zxid:0x5a txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
> [junit4:junit4]   2> 6038 T471 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
> [junit4:junit4]   2> 6039 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0006 type:create cxid:0x44 zxid:0x5b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 6049 T471 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
> [junit4:junit4]   2> 6049 T471 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
> [junit4:junit4]   2> 6049 T471 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:55544/collection1/
> [junit4:junit4]   2> 6049 T471 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
> [junit4:junit4]   2> 6050 T471 oasc.SyncStrategy.syncToMe http://127.0.0.1:55544/collection1/ has no replicas
> [junit4:junit4]   2> 6050 T471 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:55544/collection1/
> [junit4:junit4]   2> 6050 T471 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
> [junit4:junit4]   2> 6055 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0006 type:create cxid:0x4e zxid:0x5f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 6802 T452 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 6819 T470 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4]   2> 6819 T451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4]   2> 6819 T457 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
> [junit4:junit4]   2> 6830 T471 oasc.ZkController.register We are http://127.0.0.1:55544/collection1/ and leader is http://127.0.0.1:55544/collection1/
> [junit4:junit4]   2> 6830 T471 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:55544
> [junit4:junit4]   2> 6830 T471 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
> [junit4:junit4]   2> 6830 T471 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4:junit4]   2> 6830 T471 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4]   2> 6832 T471 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 6833 T429 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
> [junit4:junit4]   2> 6833 T429 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4:junit4]   2> 6834 T429 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4]   2> 7018 T429 oejs.Server.doStart jetty-8.1.8.v20121106
> [junit4:junit4]   2> 7022 T429 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:50286
> [junit4:junit4]   2> 7022 T429 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4:junit4]   2> 7023 T429 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4:junit4]   2> 7023 T429 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359882424180
> [junit4:junit4]   2> 7024 T429 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359882424180/solr.xml
> [junit4:junit4]   2> 7024 T429 oasc.CoreContainer.<init> New CoreContainer 503157312
> [junit4:junit4]   2> 7025 T429 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359882424180/'
> [junit4:junit4]   2> 7026 T429 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359882424180/'
> [junit4:junit4]   2> 7080 T429 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
> [junit4:junit4]   2> 7081 T429 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4:junit4]   2> 7081 T429 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4:junit4]   2> 7082 T429 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4:junit4]   2> 7082 T429 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4:junit4]   2> 7083 T429 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4:junit4]   2> 7083 T429 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4:junit4]   2> 7084 T429 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4:junit4]   2> 7084 T429 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4:junit4]   2> 7084 T429 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
> [junit4:junit4]   2> 7108 T429 oasc.CoreContainer.load Registering Log Listener
> [junit4:junit4]   2> 7128 T429 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:18580/solr
> [junit4:junit4]   2> 7128 T429 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4:junit4]   2> 7129 T429 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:18580 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@573410b6
> [junit4:junit4]   2> 7130 T429 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4]   2> 7130 T482 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:18580. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4]   2> 7131 T482 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:18580, initiating session
> [junit4:junit4]   2> 7132 T431 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:25631
> [junit4:junit4]   2> 7132 T431 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:25631
> [junit4:junit4]   2> 7134 T433 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9f4ef4bf0007 with negotiated timeout 20000 for client /140.211.11.196:25631
> [junit4:junit4]   2> 7134 T482 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:18580, sessionid = 0x13c9f4ef4bf0007, negotiated timeout = 20000
> [junit4:junit4]   2> 7135 T483 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@573410b6 name:ZooKeeperConnection Watcher:127.0.0.1:18580 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4]   2> 7135 T429 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4]   2> 7136 T434 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c9f4ef4bf0007
> [junit4:junit4]   2> 7154 T483 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4]   2> 7154 T429 oaz.ZooKeeper.close Session: 0x13c9f4ef4bf0007 closed
> [junit4:junit4]   2> 7154 T431 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:25631 which had sessionid 0x13c9f4ef4bf0007
> [junit4:junit4]   2> 7155 T429 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
> [junit4:junit4]   2> 7168 T429 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:18580/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@6e0e3cec
> [junit4:junit4]   2> 7170 T429 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4]   2> 7170 T484 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:18580. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4]   2> 7171 T484 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:18580, initiating session
> [junit4:junit4]   2> 7171 T431 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:13699
> [junit4:junit4]   2> 7171 T431 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:13699
> [junit4:junit4]   2> 7173 T433 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9f4ef4bf0008 with negotiated timeout 20000 for client /140.211.11.196:13699
> [junit4:junit4]   2> 7173 T484 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:18580, sessionid = 0x13c9f4ef4bf0008, negotiated timeout = 20000
> [junit4:junit4]   2> 7173 T485 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6e0e3cec name:ZooKeeperConnection Watcher:127.0.0.1:18580/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4]   2> 7174 T429 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4]   2> 7175 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0008 type:create cxid:0x1 zxid:0x6c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 7176 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0008 type:create cxid:0x2 zxid:0x6d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 7184 T429 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50286_
> [junit4:junit4]   2> 7185 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0008 type:delete cxid:0x4 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:50286_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:50286_
> [junit4:junit4]   2> 7186 T429 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50286_
> [junit4:junit4]   2> 7188 T451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
> [junit4:junit4]   2> 7189 T457 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4:junit4]   2> 7189 T470 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4:junit4]   2> 7189 T457 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
> [junit4:junit4]   2> 7189 T470 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
> [junit4:junit4]   2> 7191 T451 oascc.ZkStateReader$3.process Updating live nodes... (3)
> [junit4:junit4]   2> 7193 T429 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
> [junit4:junit4]   2> 7201 T486 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359882424180/collection1
> [junit4:junit4]   2> 7201 T486 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
> [junit4:junit4]   2> 7202 T486 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4:junit4]   2> 7202 T486 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
> [junit4:junit4]   2> 7203 T486 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359882424180/collection1/'
> [junit4:junit4]   2> 7204 T486 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359882424180/collection1/lib/classes/' to classloader
> [junit4:junit4]   2> 7204 T486 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359882424180/collection1/lib/README' to classloader
> [junit4:junit4]   2> 7241 T486 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4:junit4]   2> 7290 T486 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4:junit4]   2> 7391 T486 oass.IndexSchema.readSchema Reading Solr Schema
> [junit4:junit4]   2> 7400 T486 oass.IndexSchema.readSchema Schema name=test
> [junit4:junit4]   2> 8073 T486 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4:junit4]   2> 8084 T486 oass.IndexSchema.readSchema default search field in schema is text
> [junit4:junit4]   2> 8088 T486 oass.IndexSchema.readSchema unique key field: id
> [junit4:junit4]   2> 8102 T486 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4]   2> 8107 T486 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4]   2> 8112 T486 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4]   2> 8113 T486 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4]   2> 8115 T486 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359882424180/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty2/
> [junit4:junit4]   2> 8115 T486 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2bd1127
> [junit4:junit4]   2> 8116 T486 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
> [junit4:junit4]   2> 8116 T486 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty2 forceNew:false
> [junit4:junit4]   2> 8117 T486 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty2
> [junit4:junit4]   2> 8117 T486 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty2/index/
> [junit4:junit4]   2> 8118 T486 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty2/index' doesn't exist. Creating new index...
> [junit4:junit4]   2> 8119 T486 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty2/index forceNew:false
> [junit4:junit4]   2> 8125 T486 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@644151dc),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4]   2> 8125 T486 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
> [junit4:junit4]   2> 8126 T486 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty2/index
> [junit4:junit4]   2> 8127 T486 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
> [junit4:junit4]   2> 8129 T486 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4:junit4]   2> 8129 T486 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
> [junit4:junit4]   2> 8130 T486 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4:junit4]   2> 8130 T486 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
> [junit4:junit4]   2> 8131 T486 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4:junit4]   2> 8131 T486 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4:junit4]   2> 8131 T486 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4:junit4]   2> 8132 T486 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4:junit4]   2> 8133 T486 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4:junit4]   2> 8133 T486 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
> [junit4:junit4]   2> 8134 T486 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
> [junit4:junit4]   2> 8134 T486 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4:junit4]   2> 8135 T486 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
> [junit4:junit4]   2> 8135 T486 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
> [junit4:junit4]   2> 8135 T486 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
> [junit4:junit4]   2> 8136 T486 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4:junit4]   2> 8137 T486 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 8137 T486 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 8138 T486 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 8138 T486 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 8139 T486 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 8140 T486 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 8140 T486 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 8141 T486 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 8141 T486 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
> [junit4:junit4]   2> 8142 T486 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
> [junit4:junit4]   2> 8155 T486 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4:junit4]   2> 8160 T486 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
> [junit4:junit4]   2> 8162 T486 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
> [junit4:junit4]   2> 8164 T486 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty2
> [junit4:junit4]   2> 8166 T486 oass.SolrIndexSearcher.<init> Opening Searcher@4d1161e2 main
> [junit4:junit4]   2> 8167 T486 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4:junit4]   2> 8167 T486 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4:junit4]   2> 8168 T486 oashc.SpellCheckComponent.inform Initializing spell checkers
> [junit4:junit4]   2> 8175 T486 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
> [junit4:junit4]   2> 8229 T487 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4d1161e2 main{StandardDirectoryReader(segments_1:1)}
> [junit4:junit4]   2> 8236 T486 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4]   2> 8237 T486 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4]   2> 8326 T452 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 8326 T452 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4]   2> 	  "operation":"state",
> [junit4:junit4]   2> 	  "numShards":"2",
> [junit4:junit4]   2> 	  "shard":"shard1",
> [junit4:junit4]   2> 	  "roles":null,
> [junit4:junit4]   2> 	  "state":"active",
> [junit4:junit4]   2> 	  "core":"collection1",
> [junit4:junit4]   2> 	  "collection":"collection1",
> [junit4:junit4]   2> 	  "node_name":"127.0.0.1:55544_",
> [junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:55544"}
> [junit4:junit4]   2> 8331 T452 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4]   2> 	  "operation":"state",
> [junit4:junit4]   2> 	  "numShards":"2",
> [junit4:junit4]   2> 	  "shard":null,
> [junit4:junit4]   2> 	  "roles":null,
> [junit4:junit4]   2> 	  "state":"down",
> [junit4:junit4]   2> 	  "core":"collection1",
> [junit4:junit4]   2> 	  "collection":"collection1",
> [junit4:junit4]   2> 	  "node_name":"127.0.0.1:50286_",
> [junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:50286"}
> [junit4:junit4]   2> 8331 T452 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
> [junit4:junit4]   2> 8332 T452 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
> [junit4:junit4]   2> 8356 T457 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4]   2> 8356 T470 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4]   2> 8356 T451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4]   2> 8356 T485 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4]   2> 9239 T486 oasc.CoreContainer.registerCore registering core: collection1
> [junit4:junit4]   2> 9239 T486 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:50286 collection:collection1 shard:shard2
> [junit4:junit4]   2> 9240 T486 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
> [junit4:junit4]   2> 9281 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0008 type:delete cxid:0x42 zxid:0x7c txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
> [junit4:junit4]   2> 9282 T486 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
> [junit4:junit4]   2> 9283 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0008 type:create cxid:0x43 zxid:0x7d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 9307 T486 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
> [junit4:junit4]   2> 9308 T486 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
> [junit4:junit4]   2> 9308 T486 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:50286/collection1/
> [junit4:junit4]   2> 9308 T486 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
> [junit4:junit4]   2> 9308 T486 oasc.SyncStrategy.syncToMe http://127.0.0.1:50286/collection1/ has no replicas
> [junit4:junit4]   2> 9308 T486 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:50286/collection1/
> [junit4:junit4]   2> 9308 T486 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
> [junit4:junit4]   2> 9311 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0008 type:create cxid:0x4c zxid:0x80 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 9871 T452 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 9915 T457 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4]   2> 9915 T451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4]   2> 9915 T485 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4]   2> 9915 T470 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
> [junit4:junit4]   2> 9953 T486 oasc.ZkController.register We are http://127.0.0.1:50286/collection1/ and leader is http://127.0.0.1:50286/collection1/
> [junit4:junit4]   2> 9953 T486 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:50286
> [junit4:junit4]   2> 9953 T486 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
> [junit4:junit4]   2> 9953 T486 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4:junit4]   2> 9954 T486 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4]   2> 9955 T486 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 9956 T429 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
> [junit4:junit4]   2> 9957 T429 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4:junit4]   2> 9958 T429 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4]   2> 10083 T429 oejs.Server.doStart jetty-8.1.8.v20121106
> [junit4:junit4]   2> 10085 T429 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:40821
> [junit4:junit4]   2> 10086 T429 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4:junit4]   2> 10086 T429 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4:junit4]   2> 10087 T429 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359882427301
> [junit4:junit4]   2> 10087 T429 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359882427301/solr.xml
> [junit4:junit4]   2> 10087 T429 oasc.CoreContainer.<init> New CoreContainer 809695065
> [junit4:junit4]   2> 10088 T429 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359882427301/'
> [junit4:junit4]   2> 10088 T429 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359882427301/'
> [junit4:junit4]   2> 10125 T429 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
> [junit4:junit4]   2> 10125 T429 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4:junit4]   2> 10126 T429 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4:junit4]   2> 10126 T429 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4:junit4]   2> 10126 T429 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4:junit4]   2> 10126 T429 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4:junit4]   2> 10127 T429 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4:junit4]   2> 10127 T429 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4:junit4]   2> 10127 T429 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4:junit4]   2> 10128 T429 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
> [junit4:junit4]   2> 10143 T429 oasc.CoreContainer.load Registering Log Listener
> [junit4:junit4]   2> 10157 T429 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:18580/solr
> [junit4:junit4]   2> 10158 T429 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4:junit4]   2> 10158 T429 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:18580 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@313b2248
> [junit4:junit4]   2> 10159 T429 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4]   2> 10159 T497 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:18580. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4]   2> 10160 T497 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:18580, initiating session
> [junit4:junit4]   2> 10161 T431 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:16227
> [junit4:junit4]   2> 10162 T431 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:16227
> [junit4:junit4]   2> 10164 T433 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9f4ef4bf0009 with negotiated timeout 20000 for client /140.211.11.196:16227
> [junit4:junit4]   2> 10164 T497 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:18580, sessionid = 0x13c9f4ef4bf0009, negotiated timeout = 20000
> [junit4:junit4]   2> 10164 T498 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@313b2248 name:ZooKeeperConnection Watcher:127.0.0.1:18580 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4]   2> 10165 T429 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4]   2> 10166 T434 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c9f4ef4bf0009
> [junit4:junit4]   2> 10177 T498 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4]   2> 10177 T431 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:16227 which had sessionid 0x13c9f4ef4bf0009
> [junit4:junit4]   2> 10177 T429 oaz.ZooKeeper.close Session: 0x13c9f4ef4bf0009 closed
> [junit4:junit4]   2> 10178 T429 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
> [junit4:junit4]   2> 10193 T429 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:18580/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@249b9319
> [junit4:junit4]   2> 10194 T429 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4]   2> 10194 T499 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:18580. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4]   2> 10195 T499 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:18580, initiating session
> [junit4:junit4]   2> 10195 T431 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:43499
> [junit4:junit4]   2> 10195 T431 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:43499
> [junit4:junit4]   2> 10197 T433 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9f4ef4bf000a with negotiated timeout 20000 for client /140.211.11.196:43499
> [junit4:junit4]   2> 10197 T499 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:18580, sessionid = 0x13c9f4ef4bf000a, negotiated timeout = 20000
> [junit4:junit4]   2> 10197 T500 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@249b9319 name:ZooKeeperConnection Watcher:127.0.0.1:18580/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4]   2> 10197 T429 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4]   2> 10199 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf000a type:create cxid:0x1 zxid:0x8d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 10200 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf000a type:create cxid:0x2 zxid:0x8e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 10207 T429 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40821_
> [junit4:junit4]   2> 10208 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf000a type:delete cxid:0x4 zxid:0x8f txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:40821_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:40821_
> [junit4:junit4]   2> 10209 T429 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:40821_
> [junit4:junit4]   2> 10211 T451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
> [junit4:junit4]   2> 10212 T485 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4:junit4]   2> 10212 T470 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4:junit4]   2> 10212 T457 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4:junit4]   2> 10212 T485 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4]   2> 10212 T457 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4]   2> 10212 T470 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4]   2> 10219 T451 oascc.ZkStateReader$3.process Updating live nodes... (4)
> [junit4:junit4]   2> 10220 T429 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
> [junit4:junit4]   2> 10228 T501 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359882427301/collection1
> [junit4:junit4]   2> 10228 T501 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
> [junit4:junit4]   2> 10229 T501 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4:junit4]   2> 10229 T501 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
> [junit4:junit4]   2> 10231 T501 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359882427301/collection1/'
> [junit4:junit4]   2> 10232 T501 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359882427301/collection1/lib/classes/' to classloader
> [junit4:junit4]   2> 10232 T501 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359882427301/collection1/lib/README' to classloader
> [junit4:junit4]   2> 10286 T501 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4:junit4]   2> 10363 T501 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4:junit4]   2> 10464 T501 oass.IndexSchema.readSchema Reading Solr Schema
> [junit4:junit4]   2> 10472 T501 oass.IndexSchema.readSchema Schema name=test
> [junit4:junit4]   2> 11053 T501 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4:junit4]   2> 11061 T501 oass.IndexSchema.readSchema default search field in schema is text
> [junit4:junit4]   2> 11064 T501 oass.IndexSchema.readSchema unique key field: id
> [junit4:junit4]   2> 11077 T501 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4]   2> 11081 T501 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4]   2> 11085 T501 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4]   2> 11086 T501 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4]   2> 11087 T501 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359882427301/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3/
> [junit4:junit4]   2> 11088 T501 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2bd1127
> [junit4:junit4]   2> 11088 T501 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
> [junit4:junit4]   2> 11089 T501 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3 forceNew:false
> [junit4:junit4]   2> 11089 T501 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3
> [junit4:junit4]   2> 11089 T501 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3/index/
> [junit4:junit4]   2> 11090 T501 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3/index' doesn't exist. Creating new index...
> [junit4:junit4]   2> 11090 T501 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3/index forceNew:false
> [junit4:junit4]   2> 11095 T501 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6668ddba),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4]   2> 11095 T501 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
> [junit4:junit4]   2> 11095 T501 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3/index
> [junit4:junit4]   2> 11096 T501 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
> [junit4:junit4]   2> 11097 T501 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4:junit4]   2> 11097 T501 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
> [junit4:junit4]   2> 11098 T501 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4:junit4]   2> 11098 T501 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
> [junit4:junit4]   2> 11098 T501 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4:junit4]   2> 11098 T501 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4:junit4]   2> 11099 T501 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4:junit4]   2> 11099 T501 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4:junit4]   2> 11100 T501 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4:junit4]   2> 11100 T501 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
> [junit4:junit4]   2> 11100 T501 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
> [junit4:junit4]   2> 11101 T501 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4:junit4]   2> 11101 T501 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
> [junit4:junit4]   2> 11101 T501 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
> [junit4:junit4]   2> 11101 T501 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
> [junit4:junit4]   2> 11102 T501 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4:junit4]   2> 11102 T501 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 11103 T501 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 11103 T501 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 11103 T501 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 11104 T501 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 11104 T501 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 11105 T501 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 11105 T501 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 11105 T501 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
> [junit4:junit4]   2> 11106 T501 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
> [junit4:junit4]   2> 11114 T501 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4:junit4]   2> 11117 T501 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
> [junit4:junit4]   2> 11118 T501 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
> [junit4:junit4]   2> 11120 T501 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3
> [junit4:junit4]   2> 11121 T501 oass.SolrIndexSearcher.<init> Opening Searcher@4d1c6ebc main
> [junit4:junit4]   2> 11122 T501 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4:junit4]   2> 11122 T501 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4:junit4]   2> 11123 T501 oashc.SpellCheckComponent.inform Initializing spell checkers
> [junit4:junit4]   2> 11128 T501 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
> [junit4:junit4]   2> 11174 T502 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4d1c6ebc main{StandardDirectoryReader(segments_1:1)}
> [junit4:junit4]   2> 11178 T501 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4]   2> 11179 T501 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4]   2> 11422 T452 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 11422 T452 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4]   2> 	  "operation":"state",
> [junit4:junit4]   2> 	  "numShards":"2",
> [junit4:junit4]   2> 	  "shard":"shard2",
> [junit4:junit4]   2> 	  "roles":null,
> [junit4:junit4]   2> 	  "state":"active",
> [junit4:junit4]   2> 	  "core":"collection1",
> [junit4:junit4]   2> 	  "collection":"collection1",
> [junit4:junit4]   2> 	  "node_name":"127.0.0.1:50286_",
> [junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:50286"}
> [junit4:junit4]   2> 11426 T452 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4]   2> 	  "operation":"state",
> [junit4:junit4]   2> 	  "numShards":"2",
> [junit4:junit4]   2> 	  "shard":null,
> [junit4:junit4]   2> 	  "roles":null,
> [junit4:junit4]   2> 	  "state":"down",
> [junit4:junit4]   2> 	  "core":"collection1",
> [junit4:junit4]   2> 	  "collection":"collection1",
> [junit4:junit4]   2> 	  "node_name":"127.0.0.1:40821_",
> [junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:40821"}
> [junit4:junit4]   2> 11426 T452 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
> [junit4:junit4]   2> 11426 T452 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
> [junit4:junit4]   2> 11439 T485 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4]   2> 11439 T457 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4]   2> 11439 T500 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4]   2> 11439 T470 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4]   2> 11439 T451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
> [junit4:junit4]   2> 12181 T501 oasc.CoreContainer.registerCore registering core: collection1
> [junit4:junit4]   2> 12181 T501 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:40821 collection:collection1 shard:shard1
> [junit4:junit4]   2> 12212 T501 oasc.ZkController.register We are http://127.0.0.1:40821/collection1/ and leader is http://127.0.0.1:55544/collection1/
> [junit4:junit4]   2> 12212 T501 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:40821
> [junit4:junit4]   2> 12212 T501 oasc.ZkController.checkRecovery Core needs to recover:collection1
> [junit4:junit4]   2> 12213 T501 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
> [junit4:junit4]   2> ASYNC  NEW_CORE C31 name=collection1 org.apache.solr.core.SolrCore@69bec2a9 url=http://127.0.0.1:40821/collection1 node=127.0.0.1:40821_ C31_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:40821_, base_url=http://127.0.0.1:40821}
> [junit4:junit4]   2> 12219 T503 C31 P40821 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
> [junit4:junit4]   2> 12220 T501 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 12220 T503 C31 P40821 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
> [junit4:junit4]   2> 12221 T503 C31 P40821 oasc.ZkController.publish publishing core=collection1 state=recovering
> [junit4:junit4]   2> 12221 T503 C31 P40821 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4]   2> 12221 T429 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
> [junit4:junit4]   2> 12222 T429 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4:junit4]   2> 12222 T429 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4]   2> 12223 T503 C31 P40821 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4]   2> 12413 T429 oejs.Server.doStart jetty-8.1.8.v20121106
> [junit4:junit4]   2> 12416 T429 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:49605
> [junit4:junit4]   2> 12417 T429 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
> [junit4:junit4]   2> 12417 T429 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
> [junit4:junit4]   2> 12418 T429 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359882429572
> [junit4:junit4]   2> 12418 T429 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359882429572/solr.xml
> [junit4:junit4]   2> 12419 T429 oasc.CoreContainer.<init> New CoreContainer 337007218
> [junit4:junit4]   2> 12420 T429 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359882429572/'
> [junit4:junit4]   2> 12420 T429 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359882429572/'
> [junit4:junit4]   2> 12474 T429 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
> [junit4:junit4]   2> 12475 T429 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
> [junit4:junit4]   2> 12475 T429 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
> [junit4:junit4]   2> 12476 T429 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
> [junit4:junit4]   2> 12476 T429 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
> [junit4:junit4]   2> 12477 T429 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
> [junit4:junit4]   2> 12477 T429 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
> [junit4:junit4]   2> 12478 T429 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
> [junit4:junit4]   2> 12478 T429 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
> [junit4:junit4]   2> 12479 T429 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
> [junit4:junit4]   2> 12502 T429 oasc.CoreContainer.load Registering Log Listener
> [junit4:junit4]   2> 12521 T429 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:18580/solr
> [junit4:junit4]   2> 12521 T429 oasc.ZkController.checkChrootPath zkHost includes chroot
> [junit4:junit4]   2> 12522 T429 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:18580 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@28ca01ba
> [junit4:junit4]   2> 12523 T429 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4]   2> 12524 T513 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:18580. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4]   2> 12524 T513 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:18580, initiating session
> [junit4:junit4]   2> 12525 T431 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:28060
> [junit4:junit4]   2> 12525 T431 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:28060
> [junit4:junit4]   2> 12527 T433 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9f4ef4bf000b with negotiated timeout 20000 for client /140.211.11.196:28060
> [junit4:junit4]   2> 12527 T513 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:18580, sessionid = 0x13c9f4ef4bf000b, negotiated timeout = 20000
> [junit4:junit4]   2> 12527 T514 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@28ca01ba name:ZooKeeperConnection Watcher:127.0.0.1:18580 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4]   2> 12528 T429 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4]   2> 12529 T434 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c9f4ef4bf000b
> [junit4:junit4]   2> 12538 T514 oaz.ClientCnxn$EventThread.run EventThread shut down
> [junit4:junit4]   2> 12538 T431 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:28060 which had sessionid 0x13c9f4ef4bf000b
> [junit4:junit4]   2> 12538 T429 oaz.ZooKeeper.close Session: 0x13c9f4ef4bf000b closed
> [junit4:junit4]   2> 12539 T429 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
> [junit4:junit4]   2> 12553 T429 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:18580/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@20117c39
> [junit4:junit4]   2> 12554 T429 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
> [junit4:junit4]   2> 12554 T515 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:18580. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
> [junit4:junit4]   2> 12555 T515 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:18580, initiating session
> [junit4:junit4]   2> 12555 T431 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:28268
> [junit4:junit4]   2> 12555 T431 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:28268
> [junit4:junit4]   2> 12556 T433 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9f4ef4bf000c with negotiated timeout 20000 for client /140.211.11.196:28268
> [junit4:junit4]   2> 12556 T515 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:18580, sessionid = 0x13c9f4ef4bf000c, negotiated timeout = 20000
> [junit4:junit4]   2> 12557 T516 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@20117c39 name:ZooKeeperConnection Watcher:127.0.0.1:18580/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
> [junit4:junit4]   2> 12557 T429 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
> [junit4:junit4]   2> 12558 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf000c type:create cxid:0x1 zxid:0x9f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 12560 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf000c type:create cxid:0x2 zxid:0xa0 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
> [junit4:junit4]   2> 12567 T429 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49605_
> [junit4:junit4]   2> 12568 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf000c type:delete cxid:0x4 zxid:0xa1 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:49605_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:49605_
> [junit4:junit4]   2> 12569 T429 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49605_
> [junit4:junit4]   2> 12571 T451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4]   2> 12571 T500 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
> [junit4:junit4]   2> 12572 T470 oascc.ZkStateReader$3.process Updating live nodes... (5)
> [junit4:junit4]   2> 12572 T485 oascc.ZkStateReader$3.process Updating live nodes... (5)
> [junit4:junit4]   2> 12572 T457 oascc.ZkStateReader$3.process Updating live nodes... (5)
> [junit4:junit4]   2> 12572 T470 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 12572 T485 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 12572 T457 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 12573 T451 oascc.ZkStateReader$3.process Updating live nodes... (5)
> [junit4:junit4]   2> 12574 T500 oascc.ZkStateReader$3.process Updating live nodes... (5)
> [junit4:junit4]   2> 12581 T429 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
> [junit4:junit4]   2> 12589 T517 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359882429572/collection1
> [junit4:junit4]   2> 12589 T517 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
> [junit4:junit4]   2> 12590 T517 oasc.ZkController.createCollectionZkNode Collection zkNode exists
> [junit4:junit4]   2> 12590 T517 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
> [junit4:junit4]   2> 12592 T517 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359882429572/collection1/'
> [junit4:junit4]   2> 12593 T517 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359882429572/collection1/lib/README' to classloader
> [junit4:junit4]   2> 12594 T517 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359882429572/collection1/lib/classes/' to classloader
> [junit4:junit4]   2> 12649 T517 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
> [junit4:junit4]   2> 12721 T517 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
> [junit4:junit4]   2> 12822 T517 oass.IndexSchema.readSchema Reading Solr Schema
> [junit4:junit4]   2> 12830 T517 oass.IndexSchema.readSchema Schema name=test
> [junit4:junit4]   2> 12954 T452 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 12954 T452 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4]   2> 	  "operation":"state",
> [junit4:junit4]   2> 	  "numShards":"2",
> [junit4:junit4]   2> 	  "shard":"shard1",
> [junit4:junit4]   2> 	  "roles":null,
> [junit4:junit4]   2> 	  "state":"recovering",
> [junit4:junit4]   2> 	  "core":"collection1",
> [junit4:junit4]   2> 	  "collection":"collection1",
> [junit4:junit4]   2> 	  "node_name":"127.0.0.1:40821_",
> [junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:40821"}
> [junit4:junit4]   2> 12967 T451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 12967 T516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 12967 T485 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 12967 T500 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 12967 T470 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 12967 T457 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 13403 T517 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
> [junit4:junit4]   2> 13410 T517 oass.IndexSchema.readSchema default search field in schema is text
> [junit4:junit4]   2> 13413 T517 oass.IndexSchema.readSchema unique key field: id
> [junit4:junit4]   2> 13427 T517 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4]   2> 13431 T517 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
> [junit4:junit4]   2> 13435 T517 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4]   2> 13436 T517 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
> [junit4:junit4]   2> 13437 T517 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359882429572/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4/
> [junit4:junit4]   2> 13437 T517 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2bd1127
> [junit4:junit4]   2> 13438 T517 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
> [junit4:junit4]   2> 13439 T517 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4 forceNew:false
> [junit4:junit4]   2> 13439 T517 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4
> [junit4:junit4]   2> 13439 T517 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4/index/
> [junit4:junit4]   2> 13440 T517 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4/index' doesn't exist. Creating new index...
> [junit4:junit4]   2> 13440 T517 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4/index forceNew:false
> [junit4:junit4]   2> 13444 T517 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@327815e3),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4]   2> 13445 T517 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
> [junit4:junit4]   2> 13445 T517 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4/index
> [junit4:junit4]   2> 13445 T517 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
> [junit4:junit4]   2> 13447 T517 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
> [junit4:junit4]   2> 13447 T517 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
> [junit4:junit4]   2> 13447 T517 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
> [junit4:junit4]   2> 13448 T517 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
> [junit4:junit4]   2> 13448 T517 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
> [junit4:junit4]   2> 13448 T517 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
> [junit4:junit4]   2> 13449 T517 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
> [junit4:junit4]   2> 13449 T517 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
> [junit4:junit4]   2> 13449 T517 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
> [junit4:junit4]   2> 13450 T517 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
> [junit4:junit4]   2> 13450 T517 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
> [junit4:junit4]   2> 13451 T517 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
> [junit4:junit4]   2> 13451 T517 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
> [junit4:junit4]   2> 13451 T517 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
> [junit4:junit4]   2> 13451 T517 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
> [junit4:junit4]   2> 13452 T517 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
> [junit4:junit4]   2> 13452 T517 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 13453 T517 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 13453 T517 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 13453 T517 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 13454 T517 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 13454 T517 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 13455 T517 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 13455 T517 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
> [junit4:junit4]   2> 13455 T517 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
> [junit4:junit4]   2> 13456 T517 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
> [junit4:junit4]   2> 13464 T517 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
> [junit4:junit4]   2> 13468 T517 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
> [junit4:junit4]   2> 13469 T517 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
> [junit4:junit4]   2> 13471 T517 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4
> [junit4:junit4]   2> 13472 T517 oass.SolrIndexSearcher.<init> Opening Searcher@388bba9b main
> [junit4:junit4]   2> 13472 T517 oasu.CommitTracker.<init> Hard AutoCommit: disabled
> [junit4:junit4]   2> 13473 T517 oasu.CommitTracker.<init> Soft AutoCommit: disabled
> [junit4:junit4]   2> 13473 T517 oashc.SpellCheckComponent.inform Initializing spell checkers
> [junit4:junit4]   2> 13478 T517 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
> [junit4:junit4]   2> 13528 T518 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@388bba9b main{StandardDirectoryReader(segments_1:1)}
> [junit4:junit4]   2> 13531 T517 oasc.ZkController.publish publishing core=collection1 state=down
> [junit4:junit4]   2> 13532 T517 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4]   2> 14473 T452 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 14474 T452 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4]   2> 	  "operation":"state",
> [junit4:junit4]   2> 	  "numShards":"2",
> [junit4:junit4]   2> 	  "shard":null,
> [junit4:junit4]   2> 	  "roles":null,
> [junit4:junit4]   2> 	  "state":"down",
> [junit4:junit4]   2> 	  "core":"collection1",
> [junit4:junit4]   2> 	  "collection":"collection1",
> [junit4:junit4]   2> 	  "node_name":"127.0.0.1:49605_",
> [junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:49605"}
> [junit4:junit4]   2> 14474 T452 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
> [junit4:junit4]   2> 14474 T452 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
> [junit4:junit4]   2> 14504 T451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 14504 T485 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 14504 T500 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 14504 T470 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 14504 T516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 14504 T457 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 14534 T517 oasc.CoreContainer.registerCore registering core: collection1
> [junit4:junit4]   2> 14534 T517 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:49605 collection:collection1 shard:shard2
> [junit4:junit4]   2> 14552 T517 oasc.ZkController.register We are http://127.0.0.1:49605/collection1/ and leader is http://127.0.0.1:50286/collection1/
> [junit4:junit4]   2> 14552 T517 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49605
> [junit4:junit4]   2> 14552 T517 oasc.ZkController.checkRecovery Core needs to recover:collection1
> [junit4:junit4]   2> 14552 T517 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
> [junit4:junit4]   2> ASYNC  NEW_CORE C32 name=collection1 org.apache.solr.core.SolrCore@6234f777 url=http://127.0.0.1:49605/collection1 node=127.0.0.1:49605_ C32_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:49605_, base_url=http://127.0.0.1:49605}
> [junit4:junit4]   2> 14553 T519 C32 P49605 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
> [junit4:junit4]   2> 14554 T519 C32 P49605 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
> [junit4:junit4]   2> 14554 T517 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 14554 T519 C32 P49605 oasc.ZkController.publish publishing core=collection1 state=recovering
> [junit4:junit4]   2> 14554 T519 C32 P49605 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4]   2> 14555 T429 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
> [junit4:junit4]   2> 14556 T429 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
> [junit4:junit4]   2> 14556 T519 C32 P49605 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4]   2> 14556 T429 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4]   2> 14570 T429 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 14572 T429 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
> [junit4:junit4]   2> 14573 T429 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2>  C31_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:40821_, base_url=http://127.0.0.1:40821}
> [junit4:junit4]   2> 15277 T503 C31 P40821 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:55544/collection1/ core=collection1 - recoveringAfterStartup=true
> [junit4:junit4]   2> 15283 T503 C31 P40821 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&connTimeout=30000&socketTimeout=30000&retry=false
> [junit4:junit4]   2> 15296 T503 C31 P40821 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:40821 START replicas=[http://127.0.0.1:55544/collection1/] nUpdates=100
> [junit4:junit4]   2> 15296 T503 C31 P40821 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
> [junit4:junit4]   2> 15297 T503 C31 P40821 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
> [junit4:junit4]   2> 15297 T503 C31 P40821 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
> [junit4:junit4]   2> 15297 T503 C31 P40821 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
> [junit4:junit4]   2> 15297 T503 C31 P40821 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
> [junit4:junit4]   2> 15298 T503 C31 P40821 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:55544/collection1/. core=collection1
> [junit4:junit4]   2> 15298 T503 C31 P40821 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4]   2> ASYNC  NEW_CORE C33 name=collection1 org.apache.solr.core.SolrCore@fffb761 url=http://127.0.0.1:55544/collection1 node=127.0.0.1:55544_ C33_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:55544_, base_url=http://127.0.0.1:55544, leader=true}
> [junit4:junit4]   2> 15310 T461 C33 P55544 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
> [junit4:junit4]   2> 15324 T462 C33 P55544 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
> [junit4:junit4]   2> 15325 T462 C33 P55544 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty1
> [junit4:junit4]   2> 15328 T462 C33 P55544 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@45782220),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4]   2> 15329 T462 C33 P55544 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
> [junit4:junit4]   2> 15356 T462 C33 P55544 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@45782220),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@45782220),segFN=segments_2,generation=2,filenames=[segments_2]
> [junit4:junit4]   2> 15356 T462 C33 P55544 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
> [junit4:junit4]   2> 15357 T462 C33 P55544 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty1
> [junit4:junit4]   2> 15358 T462 C33 P55544 oass.SolrIndexSearcher.<init> Opening Searcher@2c93ddfb realtime
> [junit4:junit4]   2> 15358 T462 C33 P55544 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4]   2> 15358 T462 C33 P55544 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 45
> [junit4:junit4]   2> 15398 T503 C31 P40821 oash.ReplicationHandler.inform Commits will be reserved for  10000
> [junit4:junit4]   2> 15399 T503 C31 P40821 oascsi.HttpClientUtil.createClient Creating new http client, config:connTimeout=5000&socketTimeout=20000&allowCompression=false&maxConnections=10000&maxConnectionsPerHost=10000
> [junit4:junit4]   2> 15416 T503 C31 P40821 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
> [junit4:junit4]   2> 15419 T463 C33 P55544 oash.ReplicationHandler.inform Commits will be reserved for  10000
> [junit4:junit4]   2> 15420 T463 C33 P55544 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2 
> [junit4:junit4]   2> 15422 T503 C31 P40821 oash.SnapPuller.fetchLatestIndex Master's generation: 2
> [junit4:junit4]   2> 15422 T503 C31 P40821 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
> [junit4:junit4]   2> 15422 T503 C31 P40821 oash.SnapPuller.fetchLatestIndex Starting replication process
> [junit4:junit4]   2> 15426 T463 C33 P55544 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty1
> [junit4:junit4]   2> 15427 T463 C33 P55544 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty1/index
> [junit4:junit4]   2> 15428 T463 C33 P55544 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=2 
> [junit4:junit4]   2> 15429 T503 C31 P40821 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
> [junit4:junit4]   2> 15430 T503 C31 P40821 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3/index.20130203070712759 forceNew:false
> [junit4:junit4]   2> 15430 T503 C31 P40821 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3
> [junit4:junit4]   2> 15431 T503 C31 P40821 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3/index.20130203070712759 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2c17d192) fullCopy=true
> [junit4:junit4]   2> 15445 T463 C33 P55544 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=8 
> [junit4:junit4]   2> 15456 T503 C31 P40821 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3/index
> [junit4:junit4]   2> 15456 T503 C31 P40821 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
> [junit4:junit4]   2> 15456 T503 C31 P40821 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130203070712759
> [junit4:junit4]   2> 15458 T503 C31 P40821 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3
> [junit4:junit4]   2> 15459 T503 C31 P40821 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3
> [junit4:junit4]   2> 15459 T503 C31 P40821 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3/index/ new=./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3/index.20130203070712759
> [junit4:junit4]   2> 15461 T503 C31 P40821 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3/index.20130203070712759 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2c17d192),segFN=segments_2,generation=2,filenames=[segments_2]
> [junit4:junit4]   2> 15462 T503 C31 P40821 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
> [junit4:junit4]   2> 15464 T503 C31 P40821 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3
> [junit4:junit4]   2> 15467 T503 C31 P40821 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
> [junit4:junit4]   2> 15467 T503 C31 P40821 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
> [junit4:junit4]   2> 15467 T503 C31 P40821 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
> [junit4:junit4]   2> 15468 T503 C31 P40821 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3/index.20130203070712759
> [junit4:junit4]   2> 15469 T503 C31 P40821 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3
> [junit4:junit4]   2> 15471 T503 C31 P40821 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3/index.20130203070712759 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2c17d192),segFN=segments_2,generation=2,filenames=[segments_2]
> [junit4:junit4]   2> 15472 T503 C31 P40821 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
> [junit4:junit4]   2> 15472 T503 C31 P40821 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
> [junit4:junit4]   2> 15472 T503 C31 P40821 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3
> [junit4:junit4]   2> 15472 T503 C31 P40821 oass.SolrIndexSearcher.<init> Opening Searcher@53f25552 main
> [junit4:junit4]   2> 15474 T502 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@53f25552 main{StandardDirectoryReader(segments_2:1:nrt)}
> [junit4:junit4]   2> 15475 T502 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3/index
> [junit4:junit4]   2> 15475 T503 C31 P40821 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
> [junit4:junit4]   2> 15477 T503 C31 P40821 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3/index.20130203070712759 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2c17d192),segFN=segments_2,generation=2,filenames=[segments_2]
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3/index.20130203070712759 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2c17d192),segFN=segments_3,generation=3,filenames=[segments_3]
> [junit4:junit4]   2> 15478 T503 C31 P40821 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
> [junit4:junit4]   2> 15479 T503 C31 P40821 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3
> [junit4:junit4]   2> 15479 T503 C31 P40821 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4]   2> 15479 T503 C31 P40821 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3/index.20130203070712759
> [junit4:junit4]   2> 15479 T503 C31 P40821 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3/index
> [junit4:junit4]   2> 15479 T503 C31 P40821 oasc.RecoveryStrategy.replay No replay needed. core=collection1
> [junit4:junit4]   2> 15480 T503 C31 P40821 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
> [junit4:junit4]   2> 15480 T503 C31 P40821 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4:junit4]   2> 15480 T503 C31 P40821 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4]   2> 15488 T503 C31 P40821 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
> [junit4:junit4]   2> 15575 T429 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 16033 T452 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 16034 T452 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4]   2> 	  "operation":"state",
> [junit4:junit4]   2> 	  "numShards":"2",
> [junit4:junit4]   2> 	  "shard":"shard2",
> [junit4:junit4]   2> 	  "roles":null,
> [junit4:junit4]   2> 	  "state":"recovering",
> [junit4:junit4]   2> 	  "core":"collection1",
> [junit4:junit4]   2> 	  "collection":"collection1",
> [junit4:junit4]   2> 	  "node_name":"127.0.0.1:49605_",
> [junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:49605"}
> [junit4:junit4]   2> 16041 T452 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4]   2> 	  "operation":"state",
> [junit4:junit4]   2> 	  "numShards":"2",
> [junit4:junit4]   2> 	  "shard":"shard1",
> [junit4:junit4]   2> 	  "roles":null,
> [junit4:junit4]   2> 	  "state":"active",
> [junit4:junit4]   2> 	  "core":"collection1",
> [junit4:junit4]   2> 	  "collection":"collection1",
> [junit4:junit4]   2> 	  "node_name":"127.0.0.1:40821_",
> [junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:40821"}
> [junit4:junit4]   2> 16061 T451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 16061 T457 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 16061 T516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 16061 T470 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 16061 T500 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 16061 T485 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 16577 T429 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 17582 T429 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2>  C32_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:49605_, base_url=http://127.0.0.1:49605}
> [junit4:junit4]   2> 18576 T519 C32 P49605 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:50286/collection1/ core=collection1 - recoveringAfterStartup=true
> [junit4:junit4]   2> 18576 T519 C32 P49605 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:49605 START replicas=[http://127.0.0.1:50286/collection1/] nUpdates=100
> [junit4:junit4]   2> 18576 T519 C32 P49605 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
> [junit4:junit4]   2> 18577 T519 C32 P49605 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
> [junit4:junit4]   2> 18577 T519 C32 P49605 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
> [junit4:junit4]   2> 18577 T519 C32 P49605 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
> [junit4:junit4]   2> 18577 T519 C32 P49605 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
> [junit4:junit4]   2> 18577 T519 C32 P49605 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:50286/collection1/. core=collection1
> [junit4:junit4]   2> 18577 T519 C32 P49605 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
> [junit4:junit4]   2> 18584 T429 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> ASYNC  NEW_CORE C34 name=collection1 org.apache.solr.core.SolrCore@1930f2a url=http://127.0.0.1:50286/collection1 node=127.0.0.1:50286_ C34_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:50286_, base_url=http://127.0.0.1:50286, leader=true}
> [junit4:junit4]   2> 18589 T476 C34 P50286 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
> [junit4:junit4]   2> 18594 T477 C34 P50286 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
> [junit4:junit4]   2> 18595 T477 C34 P50286 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty2
> [junit4:junit4]   2> 18598 T477 C34 P50286 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@644151dc),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4]   2> 18598 T477 C34 P50286 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
> [junit4:junit4]   2> 18601 T477 C34 P50286 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@644151dc),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@644151dc),segFN=segments_2,generation=2,filenames=[segments_2]
> [junit4:junit4]   2> 18601 T477 C34 P50286 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
> [junit4:junit4]   2> 18602 T477 C34 P50286 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty2
> [junit4:junit4]   2> 18603 T477 C34 P50286 oass.SolrIndexSearcher.<init> Opening Searcher@691a5672 realtime
> [junit4:junit4]   2> 18603 T477 C34 P50286 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4]   2> 18603 T477 C34 P50286 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 9
> [junit4:junit4]   2> 18604 T519 C32 P49605 oash.ReplicationHandler.inform Commits will be reserved for  10000
> [junit4:junit4]   2> 18604 T519 C32 P49605 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
> [junit4:junit4]   2> 18606 T478 C34 P50286 oash.ReplicationHandler.inform Commits will be reserved for  10000
> [junit4:junit4]   2> 18607 T478 C34 P50286 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1 
> [junit4:junit4]   2> 18607 T519 C32 P49605 oash.SnapPuller.fetchLatestIndex Master's generation: 2
> [junit4:junit4]   2> 18607 T519 C32 P49605 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
> [junit4:junit4]   2> 18607 T519 C32 P49605 oash.SnapPuller.fetchLatestIndex Starting replication process
> [junit4:junit4]   2> 18610 T478 C34 P50286 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty2
> [junit4:junit4]   2> 18611 T478 C34 P50286 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty2/index
> [junit4:junit4]   2> 18611 T478 C34 P50286 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1 
> [junit4:junit4]   2> 18612 T519 C32 P49605 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
> [junit4:junit4]   2> 18612 T519 C32 P49605 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4/index.20130203070715942 forceNew:false
> [junit4:junit4]   2> 18613 T519 C32 P49605 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4
> [junit4:junit4]   2> 18613 T519 C32 P49605 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4/index.20130203070715942 lockFactory=org.apache.lucene.store.NativeFSLockFactory@185d4ea7) fullCopy=true
> [junit4:junit4]   2> 18616 T478 C34 P50286 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
> [junit4:junit4]   2> 18618 T519 C32 P49605 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4/index
> [junit4:junit4]   2> 18618 T519 C32 P49605 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
> [junit4:junit4]   2> 18619 T519 C32 P49605 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130203070715942
> [junit4:junit4]   2> 18620 T519 C32 P49605 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4
> [junit4:junit4]   2> 18620 T519 C32 P49605 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4
> [junit4:junit4]   2> 18621 T519 C32 P49605 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4/index/ new=./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4/index.20130203070715942
> [junit4:junit4]   2> 18623 T519 C32 P49605 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4/index.20130203070715942 lockFactory=org.apache.lucene.store.NativeFSLockFactory@185d4ea7),segFN=segments_2,generation=2,filenames=[segments_2]
> [junit4:junit4]   2> 18623 T519 C32 P49605 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
> [junit4:junit4]   2> 18625 T519 C32 P49605 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4
> [junit4:junit4]   2> 18625 T519 C32 P49605 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
> [junit4:junit4]   2> 18625 T519 C32 P49605 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
> [junit4:junit4]   2> 18625 T519 C32 P49605 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
> [junit4:junit4]   2> 18626 T519 C32 P49605 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4/index.20130203070715942
> [junit4:junit4]   2> 18626 T519 C32 P49605 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4
> [junit4:junit4]   2> 18629 T519 C32 P49605 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4/index.20130203070715942 lockFactory=org.apache.lucene.store.NativeFSLockFactory@185d4ea7),segFN=segments_2,generation=2,filenames=[segments_2]
> [junit4:junit4]   2> 18629 T519 C32 P49605 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
> [junit4:junit4]   2> 18629 T519 C32 P49605 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
> [junit4:junit4]   2> 18630 T519 C32 P49605 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4
> [junit4:junit4]   2> 18630 T519 C32 P49605 oass.SolrIndexSearcher.<init> Opening Searcher@44c36d6d main
> [junit4:junit4]   2> 18631 T518 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@44c36d6d main{StandardDirectoryReader(segments_2:1:nrt)}
> [junit4:junit4]   2> 18632 T518 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4/index
> [junit4:junit4]   2> 18632 T519 C32 P49605 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
> [junit4:junit4]   2> 18635 T519 C32 P49605 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4/index.20130203070715942 lockFactory=org.apache.lucene.store.NativeFSLockFactory@185d4ea7),segFN=segments_2,generation=2,filenames=[segments_2]
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4/index.20130203070715942 lockFactory=org.apache.lucene.store.NativeFSLockFactory@185d4ea7),segFN=segments_3,generation=3,filenames=[segments_3]
> [junit4:junit4]   2> 18635 T519 C32 P49605 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
> [junit4:junit4]   2> 18636 T519 C32 P49605 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4
> [junit4:junit4]   2> 18636 T519 C32 P49605 oasu.DirectUpdateHandler2.commit end_commit_flush
> [junit4:junit4]   2> 18636 T519 C32 P49605 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4/index.20130203070715942
> [junit4:junit4]   2> 18636 T519 C32 P49605 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4/index
> [junit4:junit4]   2> 18637 T519 C32 P49605 oasc.RecoveryStrategy.replay No replay needed. core=collection1
> [junit4:junit4]   2> 18637 T519 C32 P49605 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
> [junit4:junit4]   2> 18637 T519 C32 P49605 oasc.ZkController.publish publishing core=collection1 state=active
> [junit4:junit4]   2> 18637 T519 C32 P49605 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
> [junit4:junit4]   2> 18645 T519 C32 P49605 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
> [junit4:junit4]   2> 19070 T452 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 19071 T452 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
> [junit4:junit4]   2> 	  "operation":"state",
> [junit4:junit4]   2> 	  "numShards":"2",
> [junit4:junit4]   2> 	  "shard":"shard2",
> [junit4:junit4]   2> 	  "roles":null,
> [junit4:junit4]   2> 	  "state":"active",
> [junit4:junit4]   2> 	  "core":"collection1",
> [junit4:junit4]   2> 	  "collection":"collection1",
> [junit4:junit4]   2> 	  "node_name":"127.0.0.1:49605_",
> [junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:49605"}
> [junit4:junit4]   2> 19087 T451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 19088 T470 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 19088 T457 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 19088 T500 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 19088 T485 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 19088 T516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
> [junit4:junit4]   2> 19586 T429 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
> [junit4:junit4]   2> 19588 T429 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
> [junit4:junit4]   2> ASYNC  NEW_CORE C35 name=collection1 org.apache.solr.core.SolrCore@60f1108e url=http://127.0.0.1:38869/collection1 node=127.0.0.1:38869_ C35_STATE=coll:control_collection core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:38869_, base_url=http://127.0.0.1:38869, leader=true}
> [junit4:junit4]   2> 19602 T441 C35 P38869 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/control/data
> [junit4:junit4]   2> 19606 T441 C35 P38869 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
> [junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2385857),segFN=segments_1,generation=1,filenames=[segments_1]
> [junit4:junit4]   2> 19606 T441 C35 P38869 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
> [junit4:junit4]   2> 19607 T441 C35 P38869 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/control/data
> [junit4:junit4]   2> 19608 T441 C35 P38869 oass.SolrIndexSearcher.<init> Opening Searcher@299eb4d0 realtime
> [junit4:junit4]   2> 19608 T441 C35 P38869 /update {wt=javabin&version=2} {deleteByQuery=*:* (-1425940086188408832)} 0 10
> [junit4:junit4]   2> 19617 T479 C34 P50286 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty2
> [junit4:junit4]   2> 19633 T464 C33 P55544 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty1
> [junit4:junit4]   2>  C32_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:49605_, base_url=http://127.0.0.1:49605}
> [junit4:junit4]   2> 19638 T506 C32 P49605 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4
> [junit4:junit4]   2> 19639 T506 C32 P49605 /update {update.distrib=FROMLEADER&_version_=-1425940086203088896&update.from=http://127.0.0.1:50286/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1425940086203088896)} 0 2
> [junit4:junit4]   2>  C31_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:40821_, base_url=http://127.0.0.1:40821}
> [junit4:junit4]   2> 19643 T490 C31 P40821 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3
> [junit4:junit4]   2> 19643 T490 C31 P40821 /update {update.distrib=FROMLEADER&_version_=-1425940086219866112&update.from=http://127.0.0.1:55544/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1425940086219866112)} 0 2
> [junit4:junit4]   2> 19644 T464 C33 P55544 /update {update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1425940086219866112)} 0 13
> [junit4:junit4]   2> 19645 T479 C34 P50286 /update {wt=javabin&version=2} {deleteByQuery=*:* (-1425940086203088896)} 0 34
> [junit4:junit4]   2> 19654 T441 C35 P38869 /update {wt=javabin&version=2} {add=[1 (1425940086239789056)]} 0 4
> [junit4:junit4]   2> 19671 T490 C31 P40821 /update {distrib.from=http://127.0.0.1:55544/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1425940086246080512)]} 0 3
> [junit4:junit4]   2> 19672 T465 C33 P55544 /update {wt=javabin&version=2} {add=[1 (1425940086246080512)]} 0 16
> [junit4:junit4]   2> 19677 T441 C35 P38869 /update {wt=javabin&version=2} {add=[2 (1425940086266003456)]} 0 2
> [junit4:junit4]   2> 19693 T506 C32 P49605 /update {distrib.from=http://127.0.0.1:50286/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1425940086275440640)]} 0 3
> [junit4:junit4]   2> 19694 T480 C34 P50286 /update {distrib.from=http://127.0.0.1:55544/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[2 (1425940086275440640)]} 0 10
> [junit4:junit4]   2> 19694 T465 C33 P55544 /update {wt=javabin&version=2} {add=[2]} 0 15
> [junit4:junit4]   2> 19699 T441 C35 P38869 /update {wt=javabin&version=2} {add=[3 (1425940086289072128)]} 0 2
> [junit4:junit4]   2> 19706 T506 C32 P49605 /update {distrib.from=http://127.0.0.1:50286/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[3 (1425940086292217856)]} 0 1
> [junit4:junit4]   2> 19707 T479 C34 P50286 /update {wt=javabin&version=2} {add=[3 (1425940086292217856)]} 0 6
> [junit4:junit4]   2> 19712 T441 C35 P38869 /update {wt=javabin&version=2} {add=[4 (1425940086301655040)]} 0 2
> [junit4:junit4]   2> 19719 T490 C31 P40821 /update {distrib.from=http://127.0.0.1:55544/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[4 (1425940086305849344)]} 0 1
> [junit4:junit4]   2> 19720 T465 C33 P55544 /update {wt=javabin&version=2} {add=[4 (1425940086305849344)]} 0 6
> [junit4:junit4]   2> 19726 T441 C35 P38869 /update {wt=javabin&version=2} {add=[5 (1425940086316335104)]} 0 2
> [junit4:junit4]   2> 19736 T506 C32 P49605 /update {distrib.from=http://127.0.0.1:50286/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[5 (1425940086323675136)]} 0 1
> [junit4:junit4]   2> 19737 T480 C34 P50286 /update {distrib.from=http://127.0.0.1:55544/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[5 (1425940086323675136)]} 0 6
> [junit4:junit4]   2> 19738 T465 C33 P55544 /update {wt=javabin&version=2} {add=[5]} 0 10
> [junit4:junit4]   2> 19742 T441 C35 P38869 /update {wt=javabin&version=2} {add=[6 (1425940086333112320)]} 0 2
> [junit4:junit4]   2> 19749 T506 C32 P49605 /update {distrib.from=http://127.0.0.1:50286/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[6 (1425940086337306624)]} 0 1
> [junit4:junit4]   2> 19750 T479 C34 P50286 /update {wt=javabin&version=2} {add=[6 (1425940086337306624)]} 0 6
> [junit4:junit4]   2> 19755 T441 C35 P38869 /update {wt=javabin&version=2} {add=[7 (1425940086346743808)]} 0 2
> [junit4:junit4]   2> 19762 T506 C32 P49605 /update {distrib.from=http://127.0.0.1:50286/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[7 (1425940086350938112)]} 0 1
> [junit4:junit4]   2> 19763 T479 C34 P50286 /update {wt=javabin&version=2} {add=[7 (1425940086350938112)]} 0 6
> [junit4:junit4]   2> 19768 T441 C35 P38869 /update {wt=javabin&version=2} {add=[8 (1425940086361423872)]} 0 2
> [junit4:junit4]   2> 19776 T490 C31 P40821 /update {distrib.from=http://127.0.0.1:55544/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[8 (1425940086365618176)]} 0 1
> [junit4:junit4]   2> 19777 T465 C33 P55544 /update {wt=javabin&version=2} {add=[8 (1425940086365618176)]} 0 6
> [junit4:junit4]   2> 19782 T441 C35 P38869 /update {wt=javabin&version=2} {add=[9 (1425940086375055360)]} 0 2
> [junit4:junit4]   2> 19789 T506 C32 P49605 /update {distrib.from=http://127.0.0.1:50286/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[9 (1425940086379249664)]} 0 1
> [junit4:junit4]   2> 19790 T479 C34 P50286 /update {wt=javabin&version=2} {add=[9 (1425940086379249664)]} 0 6
> [junit4:junit4]   2> 19835 T441 C35 P38869 /update {wt=javabin&version=2} {add=[10 (1425940086388686848)]} 0 42
> [junit4:junit4]   2> 19844 T490 C31 P40821 /update {distrib.from=http://127.0.0.1:55544/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10 (1425940086437969920)]} 0 1
> [junit4:junit4]   2> 19845 T465 C33 P55544 /update {wt=javabin&version=2} {add=[10 (1425940086437969920)]} 0 6
> [junit4:junit4]   2> 19851 T441 C35 P38869 /update {wt=javabin&version=2} {add=[11 (1425940086447407104)]} 0 3
> [junit4:junit4]   2> 19862 T490 C31 P40821 /update {distrib.from=http://127.0.0.1:55544/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[11 (1425940086455795712)]} 0 1
> [junit4:junit4]   2> 19863 T464 C33 P55544 /update {distrib.from=http://127.0.0.1:50286/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[11 (1425940086455795712)]} 0 6
> [junit4:junit4]   2> 19863 T479 C34 P50286 /update {wt=javabin&version=2} {add=[11]} 0 9
> [junit4:junit4]   2> 19869 T441 C35 P38869 /update {wt=javabin&version=2} {add=[12 (1425940086467330048)]} 0 2
> [junit4:junit4]   2> 19880 T490 C31 P40821 /update {distrib.from=http://127.0.0.1:55544/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[12 (1425940086474670080)]} 0 1
> [junit4:junit4]   2> 19881 T464 C33 P55544 /update {distrib.from=http://127.0.0.1:50286/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[12 (1425940086474670080)]} 0 6
> [junit4:junit4]   2> 19882 T479 C34 P50286 /update {wt=javabin&version=2} {add=[12]} 0 10
> [junit4:junit4]   2> 19887 T441 C35 P38869 /update {wt=javabin&version=2} {add=[13 (1425940086486204416)]} 0 2
> [junit4:junit4]   2> 19899 T490 C31 P40821 /update {distrib.from=http://127.0.0.1:55544/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[13 (1425940086493544448)]} 0 2
> [junit4:junit4]   2> 19899 T464 C33 P55544 /update {distrib.from=http://127.0.0.1:50286/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[13 (1425940086493544448)]} 0 6
> [junit4:junit4]   2> 19900 T479 C34 P50286 /update {wt=javabin&version=2} {add=[13]} 0 10
> [junit4:junit4]   2> 19905 T441 C35 P38869 /update {wt=javabin&version=2} {add=[14 (1425940086505078784)]} 0 2
> [junit4:junit4]   2> 19916 T490 C31 P40821 /update {distrib.from=http://127.0.0.1:55544/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[14 (1425940086512418816)]} 0 1
> [junit4:junit4]   2> 19917 T464 C33 P55544 /update {distrib.from=http://127.0.0.1:50286/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[14 (1425940086512418816)]} 0 6
> [junit4:junit4]   2> 19917 T479 C34 P50286 /update {wt=javabin&version=2} {add=[14]} 0 9
> [junit4:junit4]   2> 19922 T441 C35 P38869 /update {wt=javabin&version=2} {add=[15 (1425940086522904576)]} 0 2
> [junit4:junit4]   2> 19933 T490 C31 P40821 /update {distrib.from=http://127.0.0.1:55544/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[15 (1425940086531293184)]} 0 1
> [junit4:junit4]   2> 19934 T464 C33 P55544 /update {distrib.from=http://127.0.0.1:50286/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[15 (1425940086531293184)]} 0 6
> [junit4:junit4]   2> 19934 T479 C34 P50286 /update {wt=javabin&version=2} {add=[15]} 0 9
> [junit4:junit4]   2> 19940 T441 C35 P38869 /update {wt=javabin&version=2} {add=[16 (1425940086540730368)]} 0 3
> [junit4:junit4]   2> 20010 T490 C31 P40821 /update {distrib.from=http://127.0.0.1:55544/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[16 (1425940086550167552)]} 0 29
> [junit4:junit4]   2> 20011 T464 C33 P55544 /update {distrib.from=http://127.0.0.1:50286/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[16 (1425940086550167552)]} 0 65
> [junit4:junit4]   2> 20012 T479 C34 P50286 /update {wt=javabin&version=2} {add=[16]} 0 69
> [junit4:junit4]   2> 20023 T441 C35 P38869 /update {wt=javabin&version=2} {add=[17 (1425940086624616448)]} 0
> 
> [...truncated too long message...]
> 
> te=TIMED_WAITING, group=TGRP-BasicDistributedZkTest]
> [junit4:junit4]   2> 	        at java.lang.Object.wait(Native Method)
> [junit4:junit4]   2> 	        at java.util.TimerThread.mainLoop(Timer.java:552)
> [junit4:junit4]   2> 	        at java.util.TimerThread.run(Timer.java:505)
> [junit4:junit4]   2> 	  39) Thread[id=450, name=TEST-BasicDistributedZkTest.testDistribSearch-seed#[B1AAA651BCD0851A]-SendThread(localhost:18580), state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
> [junit4:junit4]   2> 	        at sun.nio.ch.KQueueArrayWrapper.interrupt(Native Method)
> [junit4:junit4]   2> 	        at sun.nio.ch.KQueueArrayWrapper.interrupt(KQueueArrayWrapper.java:164)
> [junit4:junit4]   2> 	        at sun.nio.ch.KQueueSelectorImpl.wakeup(KQueueSelectorImpl.java:245)
> [junit4:junit4]   2> 	        at java.nio.channels.spi.AbstractSelector$1.interrupt(AbstractSelector.java:210)
> [junit4:junit4]   2> 	        at java.nio.channels.spi.AbstractSelector.begin(AbstractSelector.java:216)
> [junit4:junit4]   2> 	        at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:102)
> [junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
> [junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
> [junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)
> [junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
> [junit4:junit4]   2> 	  40) Thread[id=657, name=searcherExecutor-386-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
> [junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
> [junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> [junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
> [junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> [junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
> [junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
> [junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> [junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
> [junit4:junit4]   2> 	  41) Thread[id=502, name=searcherExecutor-270-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
> [junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
> [junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> [junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
> [junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> [junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
> [junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
> [junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> [junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
> [junit4:junit4]   2> 	  42) Thread[id=851, name=searcherExecutor-412-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
> [junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
> [junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> [junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
> [junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> [junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
> [junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
> [junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> [junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
> [junit4:junit4]   2> 	  43) Thread[id=576, name=searcherExecutor-324-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
> [junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
> [junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> [junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
> [junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> [junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
> [junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
> [junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> [junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
> [junit4:junit4]   2> NOTE: test params are: codec=Lucene42: {n_f1=PostingsFormat(name=Memory doPackFST= true), foo_b=MockVariableIntBlock(baseBlockSize=37), cat=MockFixedIntBlock(blockSize=758), foo_d=PostingsFormat(name=Lucene41WithOrds), foo_f=MockVariableIntBlock(baseBlockSize=37), n_tl1=PostingsFormat(name=Memory doPackFST= true), n_d1=MockFixedIntBlock(blockSize=758), rnd_b=PostingsFormat(name=Lucene41WithOrds), intDefault=PostingsFormat(name=Memory doPackFST= true), n_td1=PostingsFormat(name=Memory doPackFST= true), timestamp=MockFixedIntBlock(blockSize=758), id=PostingsFormat(name=Lucene41WithOrds), range_facet_sl=PostingsFormat(name=Lucene41WithOrds), range_facet_si=MockFixedIntBlock(blockSize=758), oddField_s=PostingsFormat(name=Lucene41WithOrds), sequence_i=PostingsFormat(name=Lucene41WithOrds), name=PostingsFormat(name=Lucene41WithOrds), foo_i=PostingsFormat(name=Memory doPackFST= true), regex_dup_B_s=MockFixedIntBlock(blockSize=758), multiDefault=MockFixedIntBlock(blockSize=758), n_tf1=MockFixedIntBlock(blockSize=758), n_dt1=PostingsFormat(name=Memory doPackFST= true), genre_s=PostingsFormat(name=Lucene41WithOrds), author_t=PostingsFormat(name=Memory doPackFST= true), n_ti1=PostingsFormat(name=Lucene41WithOrds), range_facet_l=MockFixedIntBlock(blockSize=758), text=MockVariableIntBlock(baseBlockSize=37), _version_=PostingsFormat(name=Memory doPackFST= true), val_i=PostingsFormat(name=Lucene41WithOrds), SubjectTerms_mfacet=PostingsFormat(name=Memory doPackFST= true), series_t=PostingsFormat(name=Memory doPackFST= true), a_t=MockFixedIntBlock(blockSize=758), n_tdt1=PostingsFormat(name=Memory doPackFST= true), regex_dup_A_s=MockVariableIntBlock(baseBlockSize=37), price=MockVariableIntBlock(baseBlockSize=37), other_tl1=PostingsFormat(name=Memory doPackFST= true), n_l1=MockFixedIntBlock(blockSize=758), a_si=PostingsFormat(name=Memory doPackFST= true), inStock=MockVariableIntBlock(baseBlockSize=37)}, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=tr, timezone=America/Bahia
> [junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=46,free=259081800,total=377487360
> [junit4:junit4]   2> NOTE: All tests run in this JVM: [ClusterStateUpdateTest, TestComponentsName, QueryElevationComponentTest, ResourceLoaderTest, SOLR749Test, TestSolrQueryParser, DistributedSpellCheckComponentTest, SpatialFilterTest, FileBasedSpellCheckerTest, TestPerFieldSimilarity, TestSolrIndexConfig, TestStressVersions, TermsComponentTest, CopyFieldTest, TestXIncludeConfig, SimpleFacetsTest, TestFaceting, JSONWriterTest, ChaosMonkeySafeLeaderTest, DirectUpdateHandlerOptimizeTest, TestWordDelimiterFilterFactory, BadComponentTest, TestBadConfig, ExternalFileFieldSortTest, FastVectorHighlighterTest, TestPHPSerializedResponseWriter, TestJoin, CircularListTest, TestFuzzyAnalyzedSuggestions, TestZkChroot, TestSystemIdResolver, BasicDistributedZkTest]
> [junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=BasicDistributedZkTest -Dtests.seed=B1AAA651BCD0851A -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=tr -Dtests.timezone=America/Bahia -Dtests.file.encoding=UTF-8
> [junit4:junit4] ERROR   0.00s J0 | BasicDistributedZkTest (suite) <<<
> [junit4:junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
> [junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([B1AAA651BCD0851A]:0)
> [junit4:junit4] Completed on J0 in 7224.12s, 1 test, 2 errors <<< FAILURES!
> 
> [...truncated 98 lines...]
> BUILD FAILED
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:346: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:39: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build.xml:178: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/common-build.xml:428: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:1176: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:841: There were test failures: 249 suites, 1038 tests, 1 suite-level error, 1 error, 128 ignored (4 assumptions)
> 
> Total time: 145 minutes 8 seconds
> Build step 'Invoke Ant' marked build as failure
> Archiving artifacts
> Recording test results
> Email was triggered for: Failure
> Sending email for trigger: Failure
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[JENKINS] Lucene-Solr-Tests-trunk-java7 - Build # 3704 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-java7/3704/

2 tests failed.
FAILED:  org.apache.solr.cloud.BasicDistributedZkTest.testDistribSearch

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([B1AAA651BCD0851A]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([B1AAA651BCD0851A]:0)




Build Log:
[...truncated 9416 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
[junit4:junit4]   2> 9981 T3 oasc.CoreContainer.finalize SEVERE CoreContainer was not shutdown prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!  instance=1207232956
[junit4:junit4]   2> 0 T429 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4]   2> 5 T429 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZkTest-1359882417336
[junit4:junit4]   2> 6 T429 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 7 T430 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 10 T430 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 10 T430 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4]   2> 10 T430 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4]   2> 11 T430 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 12 T430 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4]   2> 108 T429 oasc.ZkTestServer.run start zk server on port:18580
[junit4:junit4]   2> 108 T429 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:18580 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@36729c7c
[junit4:junit4]   2> 109 T429 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 110 T435 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:18580. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 110 T435 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:18580, initiating session
[junit4:junit4]   2> 111 T431 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:55124
[junit4:junit4]   2> 111 T431 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:55124
[junit4:junit4]   2> 111 T433 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 113 T433 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9f4ef4bf0000 with negotiated timeout 10000 for client /140.211.11.196:55124
[junit4:junit4]   2> 114 T435 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:18580, sessionid = 0x13c9f4ef4bf0000, negotiated timeout = 10000
[junit4:junit4]   2> 114 T436 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@36729c7c name:ZooKeeperConnection Watcher:127.0.0.1:18580 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 114 T429 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 115 T429 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 118 T434 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c9f4ef4bf0000
[junit4:junit4]   2> 132 T436 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 132 T429 oaz.ZooKeeper.close Session: 0x13c9f4ef4bf0000 closed
[junit4:junit4]   2> 132 T431 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:55124 which had sessionid 0x13c9f4ef4bf0000
[junit4:junit4]   2> 132 T429 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:18580/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@50242f7d
[junit4:junit4]   2> 133 T429 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 133 T437 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:18580. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 134 T437 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:18580, initiating session
[junit4:junit4]   2> 134 T431 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:17574
[junit4:junit4]   2> 134 T431 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:17574
[junit4:junit4]   2> 136 T433 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9f4ef4bf0001 with negotiated timeout 10000 for client /140.211.11.196:17574
[junit4:junit4]   2> 136 T437 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:18580, sessionid = 0x13c9f4ef4bf0001, negotiated timeout = 10000
[junit4:junit4]   2> 136 T438 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@50242f7d name:ZooKeeperConnection Watcher:127.0.0.1:18580/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 136 T429 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 137 T429 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 146 T429 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 150 T429 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 153 T429 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 161 T429 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 162 T429 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 176 T429 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 177 T429 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 282 T429 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 283 T429 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 286 T429 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 287 T429 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 297 T429 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 298 T429 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 301 T429 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 302 T429 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 306 T429 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 307 T429 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 311 T429 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 312 T429 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 315 T429 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 316 T429 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 319 T434 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c9f4ef4bf0001
[junit4:junit4]   2> 320 T438 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 320 T431 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:17574 which had sessionid 0x13c9f4ef4bf0001
[junit4:junit4]   2> 320 T429 oaz.ZooKeeper.close Session: 0x13c9f4ef4bf0001 closed
[junit4:junit4]   2> 486 T429 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 490 T429 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:38869
[junit4:junit4]   2> 491 T429 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 491 T429 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 492 T429 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359882417651
[junit4:junit4]   2> 492 T429 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359882417651/solr.xml
[junit4:junit4]   2> 493 T429 oasc.CoreContainer.<init> New CoreContainer 821516233
[junit4:junit4]   2> 494 T429 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359882417651/'
[junit4:junit4]   2> 494 T429 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359882417651/'
[junit4:junit4]   2> 548 T429 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 548 T429 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 549 T429 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 549 T429 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 550 T429 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 550 T429 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 551 T429 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 551 T429 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 551 T429 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 552 T429 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 575 T429 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 594 T429 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:18580/solr
[junit4:junit4]   2> 595 T429 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 595 T429 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:18580 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@15f502e3
[junit4:junit4]   2> 596 T429 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 597 T448 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:18580. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 597 T448 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:18580, initiating session
[junit4:junit4]   2> 598 T431 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:46186
[junit4:junit4]   2> 598 T431 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:46186
[junit4:junit4]   2> 600 T433 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9f4ef4bf0002 with negotiated timeout 20000 for client /140.211.11.196:46186
[junit4:junit4]   2> 600 T448 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:18580, sessionid = 0x13c9f4ef4bf0002, negotiated timeout = 20000
[junit4:junit4]   2> 600 T449 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@15f502e3 name:ZooKeeperConnection Watcher:127.0.0.1:18580 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 601 T429 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 602 T434 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c9f4ef4bf0002
[junit4:junit4]   2> 603 T449 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 603 T431 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:46186 which had sessionid 0x13c9f4ef4bf0002
[junit4:junit4]   2> 603 T429 oaz.ZooKeeper.close Session: 0x13c9f4ef4bf0002 closed
[junit4:junit4]   2> 604 T429 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 617 T429 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:18580/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@8682e3
[junit4:junit4]   2> 618 T429 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 618 T450 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:18580. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 619 T450 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:18580, initiating session
[junit4:junit4]   2> 619 T431 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:23653
[junit4:junit4]   2> 619 T431 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:23653
[junit4:junit4]   2> 621 T433 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9f4ef4bf0003 with negotiated timeout 20000 for client /140.211.11.196:23653
[junit4:junit4]   2> 621 T450 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:18580, sessionid = 0x13c9f4ef4bf0003, negotiated timeout = 20000
[junit4:junit4]   2> 621 T451 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@8682e3 name:ZooKeeperConnection Watcher:127.0.0.1:18580/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 621 T429 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 624 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 632 T429 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 635 T429 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38869_
[junit4:junit4]   2> 635 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0003 type:delete cxid:0x7 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:38869_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:38869_
[junit4:junit4]   2> 636 T429 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:38869_
[junit4:junit4]   2> 646 T429 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 659 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0003 type:delete cxid:0x15 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 660 T429 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 663 T429 oasc.Overseer.start Overseer (id=89121254102990851-127.0.0.1:38869_-n_0000000000) starting
[junit4:junit4]   2> 664 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0003 type:create cxid:0x1a zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 671 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0003 type:create cxid:0x1b zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 672 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0003 type:create cxid:0x1c zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 679 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0003 type:create cxid:0x1d zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 680 T453 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 681 T429 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 691 T429 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 694 T452 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 699 T454 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359882417651/collection1
[junit4:junit4]   2> 699 T454 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 700 T454 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 700 T454 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 702 T454 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359882417651/collection1/'
[junit4:junit4]   2> 703 T454 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359882417651/collection1/lib/README' to classloader
[junit4:junit4]   2> 703 T454 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359882417651/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 755 T454 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 826 T454 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 927 T454 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 935 T454 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 1558 T454 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1566 T454 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1569 T454 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1583 T454 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1587 T454 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1590 T454 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1592 T454 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1593 T454 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359882417651/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/control/data/
[junit4:junit4]   2> 1593 T454 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2bd1127
[junit4:junit4]   2> 1593 T454 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 1594 T454 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/control/data forceNew:false
[junit4:junit4]   2> 1595 T454 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/control/data
[junit4:junit4]   2> 1595 T454 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/control/data/index/
[junit4:junit4]   2> 1595 T454 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1596 T454 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/control/data/index forceNew:false
[junit4:junit4]   2> 1600 T454 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2385857),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 1601 T454 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 1601 T454 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/control/data/index
[junit4:junit4]   2> 1601 T454 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 1603 T454 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1603 T454 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 1603 T454 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1603 T454 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 1604 T454 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1604 T454 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1604 T454 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1605 T454 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1605 T454 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1605 T454 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 1606 T454 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 1606 T454 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1606 T454 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 1606 T454 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 1607 T454 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 1607 T454 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1607 T454 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1608 T454 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1608 T454 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1608 T454 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1608 T454 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1609 T454 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1609 T454 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1610 T454 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1610 T454 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 1610 T454 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 1618 T454 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1621 T454 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 1622 T454 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 1624 T454 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/control/data
[junit4:junit4]   2> 1625 T454 oass.SolrIndexSearcher.<init> Opening Searcher@a3807eb main
[junit4:junit4]   2> 1626 T454 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1626 T454 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1627 T454 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 1643 T454 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 1697 T455 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@a3807eb main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1700 T454 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 1700 T454 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 1701 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0003 type:create cxid:0x50 zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 2198 T452 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2199 T452 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:38869_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:38869"}
[junit4:junit4]   2> 2199 T452 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4]   2> 2204 T452 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 2205 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0003 type:create cxid:0x58 zxid:0x2c txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 2227 T451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 2713 T454 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 2713 T454 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:38869 collection:control_collection shard:shard1
[junit4:junit4]   2> 2714 T454 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4]   2> 2732 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0003 type:delete cxid:0x72 zxid:0x36 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4]   2> 2733 T454 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 2734 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0003 type:create cxid:0x73 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2736 T454 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 2736 T454 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 2736 T454 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:38869/collection1/
[junit4:junit4]   2> 2737 T454 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 2737 T454 oasc.SyncStrategy.syncToMe http://127.0.0.1:38869/collection1/ has no replicas
[junit4:junit4]   2> 2737 T454 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:38869/collection1/
[junit4:junit4]   2> 2738 T454 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4]   2> 2742 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0003 type:create cxid:0x7d zxid:0x3b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3733 T452 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3756 T451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 3765 T454 oasc.ZkController.register We are http://127.0.0.1:38869/collection1/ and leader is http://127.0.0.1:38869/collection1/
[junit4:junit4]   2> 3765 T454 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:38869
[junit4:junit4]   2> 3765 T454 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 3765 T454 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 3765 T454 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 3768 T454 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3769 T429 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 3769 T429 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 3770 T429 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 3784 T429 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 3794 T429 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:18580/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@4ecdac83
[junit4:junit4]   2> 3795 T429 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 3795 T456 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:18580. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 3796 T456 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:18580, initiating session
[junit4:junit4]   2> 3796 T431 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:48743
[junit4:junit4]   2> 3796 T431 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:48743
[junit4:junit4]   2> 3799 T433 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9f4ef4bf0004 with negotiated timeout 10000 for client /140.211.11.196:48743
[junit4:junit4]   2> 3799 T456 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:18580, sessionid = 0x13c9f4ef4bf0004, negotiated timeout = 10000
[junit4:junit4]   2> 3799 T457 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4ecdac83 name:ZooKeeperConnection Watcher:127.0.0.1:18580/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 3799 T429 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 3800 T429 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 3804 T429 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 3935 T429 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 3938 T429 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:55544
[junit4:junit4]   2> 3939 T429 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 3939 T429 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 3939 T429 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359882421134
[junit4:junit4]   2> 3940 T429 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359882421134/solr.xml
[junit4:junit4]   2> 3940 T429 oasc.CoreContainer.<init> New CoreContainer 123419126
[junit4:junit4]   2> 3940 T429 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359882421134/'
[junit4:junit4]   2> 3941 T429 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359882421134/'
[junit4:junit4]   2> 3976 T429 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 3976 T429 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 3977 T429 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 3977 T429 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 3977 T429 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 3978 T429 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 3978 T429 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 3978 T429 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 3978 T429 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 3979 T429 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 3993 T429 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 4007 T429 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:18580/solr
[junit4:junit4]   2> 4007 T429 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 4007 T429 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:18580 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@34fccead
[junit4:junit4]   2> 4008 T429 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4009 T467 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:18580. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4009 T467 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:18580, initiating session
[junit4:junit4]   2> 4009 T431 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:49421
[junit4:junit4]   2> 4010 T431 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:49421
[junit4:junit4]   2> 4011 T433 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9f4ef4bf0005 with negotiated timeout 20000 for client /140.211.11.196:49421
[junit4:junit4]   2> 4011 T467 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:18580, sessionid = 0x13c9f4ef4bf0005, negotiated timeout = 20000
[junit4:junit4]   2> 4011 T468 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@34fccead name:ZooKeeperConnection Watcher:127.0.0.1:18580 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4011 T429 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4012 T434 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c9f4ef4bf0005
[junit4:junit4]   2> 4014 T431 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:49421 which had sessionid 0x13c9f4ef4bf0005
[junit4:junit4]   2> 4014 T468 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 4013 T429 oaz.ZooKeeper.close Session: 0x13c9f4ef4bf0005 closed
[junit4:junit4]   2> 4014 T429 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 4022 T429 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:18580/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@72cd6a83
[junit4:junit4]   2> 4023 T429 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4023 T469 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:18580. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4023 T469 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:18580, initiating session
[junit4:junit4]   2> 4023 T431 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:43721
[junit4:junit4]   2> 4024 T431 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:43721
[junit4:junit4]   2> 4025 T433 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9f4ef4bf0006 with negotiated timeout 20000 for client /140.211.11.196:43721
[junit4:junit4]   2> 4025 T469 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:18580, sessionid = 0x13c9f4ef4bf0006, negotiated timeout = 20000
[junit4:junit4]   2> 4025 T470 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@72cd6a83 name:ZooKeeperConnection Watcher:127.0.0.1:18580/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4025 T429 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4026 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0006 type:create cxid:0x1 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4027 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0006 type:create cxid:0x2 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4036 T429 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55544_
[junit4:junit4]   2> 4037 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0006 type:delete cxid:0x4 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:55544_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:55544_
[junit4:junit4]   2> 4038 T429 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:55544_
[junit4:junit4]   2> 4040 T451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 4041 T457 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 4042 T451 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 4049 T429 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 4055 T471 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359882421134/collection1
[junit4:junit4]   2> 4056 T471 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 4056 T471 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 4056 T471 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 4058 T471 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359882421134/collection1/'
[junit4:junit4]   2> 4059 T471 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359882421134/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 4059 T471 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359882421134/collection1/lib/README' to classloader
[junit4:junit4]   2> 4099 T471 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 4151 T471 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 4252 T471 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 4258 T471 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 4808 T471 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 4821 T471 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 4825 T471 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 4842 T471 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 4848 T471 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 4853 T471 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 4855 T471 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 4857 T471 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359882421134/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty1/
[junit4:junit4]   2> 4857 T471 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2bd1127
[junit4:junit4]   2> 4858 T471 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 4858 T471 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty1 forceNew:false
[junit4:junit4]   2> 4859 T471 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty1
[junit4:junit4]   2> 4859 T471 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty1/index/
[junit4:junit4]   2> 4860 T471 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 4861 T471 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty1/index forceNew:false
[junit4:junit4]   2> 4868 T471 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@45782220),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 4868 T471 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 4869 T471 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty1/index
[junit4:junit4]   2> 4869 T471 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 4872 T471 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 4872 T471 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 4872 T471 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 4873 T471 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 4873 T471 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 4874 T471 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 4874 T471 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 4875 T471 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 4875 T471 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 4876 T471 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 4876 T471 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 4877 T471 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 4877 T471 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 4878 T471 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 4878 T471 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 4879 T471 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 4879 T471 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4880 T471 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4880 T471 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4881 T471 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4881 T471 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4882 T471 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4883 T471 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4883 T471 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4884 T471 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 4884 T471 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 4897 T471 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 4902 T471 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 4903 T471 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 4906 T471 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty1
[junit4:junit4]   2> 4908 T471 oass.SolrIndexSearcher.<init> Opening Searcher@3b6be770 main
[junit4:junit4]   2> 4909 T471 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 4909 T471 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 4909 T471 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 4917 T471 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 4983 T472 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3b6be770 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 4989 T471 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 4989 T471 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 5264 T452 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 5265 T452 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:38869_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:38869"}
[junit4:junit4]   2> 5272 T452 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:55544_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:55544"}
[junit4:junit4]   2> 5272 T452 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4]   2> 5273 T452 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 5287 T451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5287 T470 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5288 T457 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5992 T471 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 5992 T471 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:55544 collection:collection1 shard:shard1
[junit4:junit4]   2> 5993 T471 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 6037 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0006 type:delete cxid:0x43 zxid:0x5a txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 6038 T471 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 6039 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0006 type:create cxid:0x44 zxid:0x5b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6049 T471 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 6049 T471 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 6049 T471 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:55544/collection1/
[junit4:junit4]   2> 6049 T471 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 6050 T471 oasc.SyncStrategy.syncToMe http://127.0.0.1:55544/collection1/ has no replicas
[junit4:junit4]   2> 6050 T471 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:55544/collection1/
[junit4:junit4]   2> 6050 T471 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 6055 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0006 type:create cxid:0x4e zxid:0x5f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6802 T452 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6819 T470 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6819 T451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6819 T457 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6830 T471 oasc.ZkController.register We are http://127.0.0.1:55544/collection1/ and leader is http://127.0.0.1:55544/collection1/
[junit4:junit4]   2> 6830 T471 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:55544
[junit4:junit4]   2> 6830 T471 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 6830 T471 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 6830 T471 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 6832 T471 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6833 T429 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 6833 T429 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 6834 T429 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 7018 T429 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 7022 T429 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:50286
[junit4:junit4]   2> 7022 T429 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 7023 T429 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 7023 T429 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359882424180
[junit4:junit4]   2> 7024 T429 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359882424180/solr.xml
[junit4:junit4]   2> 7024 T429 oasc.CoreContainer.<init> New CoreContainer 503157312
[junit4:junit4]   2> 7025 T429 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359882424180/'
[junit4:junit4]   2> 7026 T429 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359882424180/'
[junit4:junit4]   2> 7080 T429 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 7081 T429 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 7081 T429 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 7082 T429 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 7082 T429 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 7083 T429 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 7083 T429 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 7084 T429 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 7084 T429 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 7084 T429 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 7108 T429 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 7128 T429 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:18580/solr
[junit4:junit4]   2> 7128 T429 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 7129 T429 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:18580 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@573410b6
[junit4:junit4]   2> 7130 T429 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 7130 T482 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:18580. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 7131 T482 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:18580, initiating session
[junit4:junit4]   2> 7132 T431 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:25631
[junit4:junit4]   2> 7132 T431 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:25631
[junit4:junit4]   2> 7134 T433 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9f4ef4bf0007 with negotiated timeout 20000 for client /140.211.11.196:25631
[junit4:junit4]   2> 7134 T482 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:18580, sessionid = 0x13c9f4ef4bf0007, negotiated timeout = 20000
[junit4:junit4]   2> 7135 T483 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@573410b6 name:ZooKeeperConnection Watcher:127.0.0.1:18580 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 7135 T429 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 7136 T434 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c9f4ef4bf0007
[junit4:junit4]   2> 7154 T483 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 7154 T429 oaz.ZooKeeper.close Session: 0x13c9f4ef4bf0007 closed
[junit4:junit4]   2> 7154 T431 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:25631 which had sessionid 0x13c9f4ef4bf0007
[junit4:junit4]   2> 7155 T429 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 7168 T429 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:18580/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@6e0e3cec
[junit4:junit4]   2> 7170 T429 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 7170 T484 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:18580. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 7171 T484 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:18580, initiating session
[junit4:junit4]   2> 7171 T431 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:13699
[junit4:junit4]   2> 7171 T431 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:13699
[junit4:junit4]   2> 7173 T433 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9f4ef4bf0008 with negotiated timeout 20000 for client /140.211.11.196:13699
[junit4:junit4]   2> 7173 T484 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:18580, sessionid = 0x13c9f4ef4bf0008, negotiated timeout = 20000
[junit4:junit4]   2> 7173 T485 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6e0e3cec name:ZooKeeperConnection Watcher:127.0.0.1:18580/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 7174 T429 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 7175 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0008 type:create cxid:0x1 zxid:0x6c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7176 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0008 type:create cxid:0x2 zxid:0x6d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7184 T429 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50286_
[junit4:junit4]   2> 7185 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0008 type:delete cxid:0x4 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:50286_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:50286_
[junit4:junit4]   2> 7186 T429 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50286_
[junit4:junit4]   2> 7188 T451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7189 T457 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7189 T470 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7189 T457 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 7189 T470 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 7191 T451 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7193 T429 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 7201 T486 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359882424180/collection1
[junit4:junit4]   2> 7201 T486 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 7202 T486 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 7202 T486 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 7203 T486 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359882424180/collection1/'
[junit4:junit4]   2> 7204 T486 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359882424180/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 7204 T486 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359882424180/collection1/lib/README' to classloader
[junit4:junit4]   2> 7241 T486 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 7290 T486 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 7391 T486 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 7400 T486 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 8073 T486 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 8084 T486 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 8088 T486 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 8102 T486 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8107 T486 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8112 T486 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8113 T486 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8115 T486 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359882424180/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty2/
[junit4:junit4]   2> 8115 T486 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2bd1127
[junit4:junit4]   2> 8116 T486 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 8116 T486 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty2 forceNew:false
[junit4:junit4]   2> 8117 T486 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty2
[junit4:junit4]   2> 8117 T486 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty2/index/
[junit4:junit4]   2> 8118 T486 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 8119 T486 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty2/index forceNew:false
[junit4:junit4]   2> 8125 T486 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@644151dc),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 8125 T486 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 8126 T486 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty2/index
[junit4:junit4]   2> 8127 T486 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 8129 T486 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 8129 T486 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 8130 T486 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 8130 T486 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 8131 T486 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 8131 T486 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 8131 T486 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 8132 T486 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 8133 T486 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 8133 T486 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 8134 T486 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 8134 T486 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 8135 T486 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 8135 T486 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 8135 T486 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 8136 T486 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 8137 T486 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8137 T486 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8138 T486 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8138 T486 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8139 T486 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8140 T486 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8140 T486 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8141 T486 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8141 T486 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 8142 T486 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 8155 T486 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 8160 T486 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 8162 T486 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 8164 T486 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty2
[junit4:junit4]   2> 8166 T486 oass.SolrIndexSearcher.<init> Opening Searcher@4d1161e2 main
[junit4:junit4]   2> 8167 T486 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 8167 T486 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 8168 T486 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 8175 T486 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 8229 T487 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4d1161e2 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 8236 T486 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 8237 T486 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 8326 T452 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8326 T452 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:55544_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:55544"}
[junit4:junit4]   2> 8331 T452 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:50286_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:50286"}
[junit4:junit4]   2> 8331 T452 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 8332 T452 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 8356 T457 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8356 T470 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8356 T451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8356 T485 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9239 T486 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 9239 T486 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:50286 collection:collection1 shard:shard2
[junit4:junit4]   2> 9240 T486 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4]   2> 9281 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0008 type:delete cxid:0x42 zxid:0x7c txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
[junit4:junit4]   2> 9282 T486 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 9283 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0008 type:create cxid:0x43 zxid:0x7d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9307 T486 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 9308 T486 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 9308 T486 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:50286/collection1/
[junit4:junit4]   2> 9308 T486 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 9308 T486 oasc.SyncStrategy.syncToMe http://127.0.0.1:50286/collection1/ has no replicas
[junit4:junit4]   2> 9308 T486 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:50286/collection1/
[junit4:junit4]   2> 9308 T486 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4]   2> 9311 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf0008 type:create cxid:0x4c zxid:0x80 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9871 T452 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9915 T457 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9915 T451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9915 T485 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9915 T470 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9953 T486 oasc.ZkController.register We are http://127.0.0.1:50286/collection1/ and leader is http://127.0.0.1:50286/collection1/
[junit4:junit4]   2> 9953 T486 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:50286
[junit4:junit4]   2> 9953 T486 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 9953 T486 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 9954 T486 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 9955 T486 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9956 T429 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 9957 T429 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 9958 T429 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 10083 T429 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 10085 T429 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:40821
[junit4:junit4]   2> 10086 T429 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 10086 T429 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 10087 T429 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359882427301
[junit4:junit4]   2> 10087 T429 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359882427301/solr.xml
[junit4:junit4]   2> 10087 T429 oasc.CoreContainer.<init> New CoreContainer 809695065
[junit4:junit4]   2> 10088 T429 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359882427301/'
[junit4:junit4]   2> 10088 T429 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359882427301/'
[junit4:junit4]   2> 10125 T429 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 10125 T429 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 10126 T429 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 10126 T429 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 10126 T429 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 10126 T429 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 10127 T429 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 10127 T429 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 10127 T429 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 10128 T429 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 10143 T429 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 10157 T429 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:18580/solr
[junit4:junit4]   2> 10158 T429 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 10158 T429 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:18580 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@313b2248
[junit4:junit4]   2> 10159 T429 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 10159 T497 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:18580. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 10160 T497 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:18580, initiating session
[junit4:junit4]   2> 10161 T431 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:16227
[junit4:junit4]   2> 10162 T431 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:16227
[junit4:junit4]   2> 10164 T433 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9f4ef4bf0009 with negotiated timeout 20000 for client /140.211.11.196:16227
[junit4:junit4]   2> 10164 T497 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:18580, sessionid = 0x13c9f4ef4bf0009, negotiated timeout = 20000
[junit4:junit4]   2> 10164 T498 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@313b2248 name:ZooKeeperConnection Watcher:127.0.0.1:18580 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 10165 T429 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 10166 T434 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c9f4ef4bf0009
[junit4:junit4]   2> 10177 T498 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 10177 T431 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:16227 which had sessionid 0x13c9f4ef4bf0009
[junit4:junit4]   2> 10177 T429 oaz.ZooKeeper.close Session: 0x13c9f4ef4bf0009 closed
[junit4:junit4]   2> 10178 T429 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 10193 T429 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:18580/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@249b9319
[junit4:junit4]   2> 10194 T429 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 10194 T499 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:18580. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 10195 T499 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:18580, initiating session
[junit4:junit4]   2> 10195 T431 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:43499
[junit4:junit4]   2> 10195 T431 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:43499
[junit4:junit4]   2> 10197 T433 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9f4ef4bf000a with negotiated timeout 20000 for client /140.211.11.196:43499
[junit4:junit4]   2> 10197 T499 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:18580, sessionid = 0x13c9f4ef4bf000a, negotiated timeout = 20000
[junit4:junit4]   2> 10197 T500 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@249b9319 name:ZooKeeperConnection Watcher:127.0.0.1:18580/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 10197 T429 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 10199 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf000a type:create cxid:0x1 zxid:0x8d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 10200 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf000a type:create cxid:0x2 zxid:0x8e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 10207 T429 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40821_
[junit4:junit4]   2> 10208 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf000a type:delete cxid:0x4 zxid:0x8f txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:40821_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:40821_
[junit4:junit4]   2> 10209 T429 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:40821_
[junit4:junit4]   2> 10211 T451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10212 T485 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 10212 T470 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 10212 T457 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 10212 T485 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 10212 T457 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 10212 T470 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 10219 T451 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 10220 T429 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 10228 T501 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359882427301/collection1
[junit4:junit4]   2> 10228 T501 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 10229 T501 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 10229 T501 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 10231 T501 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359882427301/collection1/'
[junit4:junit4]   2> 10232 T501 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359882427301/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 10232 T501 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359882427301/collection1/lib/README' to classloader
[junit4:junit4]   2> 10286 T501 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 10363 T501 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 10464 T501 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 10472 T501 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 11053 T501 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 11061 T501 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 11064 T501 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 11077 T501 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 11081 T501 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 11085 T501 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 11086 T501 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 11087 T501 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359882427301/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3/
[junit4:junit4]   2> 11088 T501 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2bd1127
[junit4:junit4]   2> 11088 T501 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 11089 T501 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3 forceNew:false
[junit4:junit4]   2> 11089 T501 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3
[junit4:junit4]   2> 11089 T501 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3/index/
[junit4:junit4]   2> 11090 T501 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 11090 T501 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3/index forceNew:false
[junit4:junit4]   2> 11095 T501 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6668ddba),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 11095 T501 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 11095 T501 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3/index
[junit4:junit4]   2> 11096 T501 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 11097 T501 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 11097 T501 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 11098 T501 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 11098 T501 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 11098 T501 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 11098 T501 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 11099 T501 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 11099 T501 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 11100 T501 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 11100 T501 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 11100 T501 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 11101 T501 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 11101 T501 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 11101 T501 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 11101 T501 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 11102 T501 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 11102 T501 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11103 T501 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11103 T501 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11103 T501 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11104 T501 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11104 T501 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11105 T501 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11105 T501 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11105 T501 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 11106 T501 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 11114 T501 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 11117 T501 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 11118 T501 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 11120 T501 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3
[junit4:junit4]   2> 11121 T501 oass.SolrIndexSearcher.<init> Opening Searcher@4d1c6ebc main
[junit4:junit4]   2> 11122 T501 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 11122 T501 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 11123 T501 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 11128 T501 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 11174 T502 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4d1c6ebc main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 11178 T501 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 11179 T501 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 11422 T452 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11422 T452 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:50286_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:50286"}
[junit4:junit4]   2> 11426 T452 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:40821_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:40821"}
[junit4:junit4]   2> 11426 T452 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 11426 T452 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 11439 T485 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11439 T457 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11439 T500 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11439 T470 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11439 T451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12181 T501 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 12181 T501 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:40821 collection:collection1 shard:shard1
[junit4:junit4]   2> 12212 T501 oasc.ZkController.register We are http://127.0.0.1:40821/collection1/ and leader is http://127.0.0.1:55544/collection1/
[junit4:junit4]   2> 12212 T501 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:40821
[junit4:junit4]   2> 12212 T501 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 12213 T501 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C31 name=collection1 org.apache.solr.core.SolrCore@69bec2a9 url=http://127.0.0.1:40821/collection1 node=127.0.0.1:40821_ C31_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:40821_, base_url=http://127.0.0.1:40821}
[junit4:junit4]   2> 12219 T503 C31 P40821 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 12220 T501 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12220 T503 C31 P40821 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 12221 T503 C31 P40821 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 12221 T503 C31 P40821 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 12221 T429 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 12222 T429 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 12222 T429 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 12223 T503 C31 P40821 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 12413 T429 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 12416 T429 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:49605
[junit4:junit4]   2> 12417 T429 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 12417 T429 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 12418 T429 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359882429572
[junit4:junit4]   2> 12418 T429 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359882429572/solr.xml
[junit4:junit4]   2> 12419 T429 oasc.CoreContainer.<init> New CoreContainer 337007218
[junit4:junit4]   2> 12420 T429 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359882429572/'
[junit4:junit4]   2> 12420 T429 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359882429572/'
[junit4:junit4]   2> 12474 T429 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 12475 T429 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 12475 T429 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 12476 T429 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 12476 T429 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 12477 T429 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 12477 T429 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 12478 T429 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 12478 T429 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 12479 T429 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 12502 T429 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 12521 T429 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:18580/solr
[junit4:junit4]   2> 12521 T429 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 12522 T429 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:18580 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@28ca01ba
[junit4:junit4]   2> 12523 T429 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 12524 T513 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:18580. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 12524 T513 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:18580, initiating session
[junit4:junit4]   2> 12525 T431 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:28060
[junit4:junit4]   2> 12525 T431 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:28060
[junit4:junit4]   2> 12527 T433 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9f4ef4bf000b with negotiated timeout 20000 for client /140.211.11.196:28060
[junit4:junit4]   2> 12527 T513 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:18580, sessionid = 0x13c9f4ef4bf000b, negotiated timeout = 20000
[junit4:junit4]   2> 12527 T514 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@28ca01ba name:ZooKeeperConnection Watcher:127.0.0.1:18580 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 12528 T429 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 12529 T434 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c9f4ef4bf000b
[junit4:junit4]   2> 12538 T514 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 12538 T431 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:28060 which had sessionid 0x13c9f4ef4bf000b
[junit4:junit4]   2> 12538 T429 oaz.ZooKeeper.close Session: 0x13c9f4ef4bf000b closed
[junit4:junit4]   2> 12539 T429 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 12553 T429 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:18580/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@20117c39
[junit4:junit4]   2> 12554 T429 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 12554 T515 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:18580. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 12555 T515 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:18580, initiating session
[junit4:junit4]   2> 12555 T431 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:28268
[junit4:junit4]   2> 12555 T431 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:28268
[junit4:junit4]   2> 12556 T433 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9f4ef4bf000c with negotiated timeout 20000 for client /140.211.11.196:28268
[junit4:junit4]   2> 12556 T515 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:18580, sessionid = 0x13c9f4ef4bf000c, negotiated timeout = 20000
[junit4:junit4]   2> 12557 T516 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@20117c39 name:ZooKeeperConnection Watcher:127.0.0.1:18580/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 12557 T429 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 12558 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf000c type:create cxid:0x1 zxid:0x9f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12560 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf000c type:create cxid:0x2 zxid:0xa0 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12567 T429 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49605_
[junit4:junit4]   2> 12568 T434 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9f4ef4bf000c type:delete cxid:0x4 zxid:0xa1 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:49605_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:49605_
[junit4:junit4]   2> 12569 T429 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49605_
[junit4:junit4]   2> 12571 T451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12571 T500 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12572 T470 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 12572 T485 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 12572 T457 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 12572 T470 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 12572 T485 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 12572 T457 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 12573 T451 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 12574 T500 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 12581 T429 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 12589 T517 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359882429572/collection1
[junit4:junit4]   2> 12589 T517 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 12590 T517 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 12590 T517 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 12592 T517 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359882429572/collection1/'
[junit4:junit4]   2> 12593 T517 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359882429572/collection1/lib/README' to classloader
[junit4:junit4]   2> 12594 T517 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359882429572/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 12649 T517 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 12721 T517 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 12822 T517 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 12830 T517 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 12954 T452 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12954 T452 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:40821_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:40821"}
[junit4:junit4]   2> 12967 T451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 12967 T516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 12967 T485 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 12967 T500 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 12967 T470 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 12967 T457 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 13403 T517 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 13410 T517 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 13413 T517 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 13427 T517 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 13431 T517 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 13435 T517 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 13436 T517 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 13437 T517 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359882429572/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4/
[junit4:junit4]   2> 13437 T517 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2bd1127
[junit4:junit4]   2> 13438 T517 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 13439 T517 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4 forceNew:false
[junit4:junit4]   2> 13439 T517 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4
[junit4:junit4]   2> 13439 T517 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4/index/
[junit4:junit4]   2> 13440 T517 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 13440 T517 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4/index forceNew:false
[junit4:junit4]   2> 13444 T517 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@327815e3),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 13445 T517 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 13445 T517 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4/index
[junit4:junit4]   2> 13445 T517 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 13447 T517 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 13447 T517 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 13447 T517 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 13448 T517 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 13448 T517 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 13448 T517 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 13449 T517 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 13449 T517 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 13449 T517 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 13450 T517 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 13450 T517 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 13451 T517 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 13451 T517 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 13451 T517 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 13451 T517 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 13452 T517 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 13452 T517 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13453 T517 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13453 T517 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13453 T517 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13454 T517 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13454 T517 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13455 T517 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13455 T517 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13455 T517 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 13456 T517 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 13464 T517 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 13468 T517 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 13469 T517 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 13471 T517 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4
[junit4:junit4]   2> 13472 T517 oass.SolrIndexSearcher.<init> Opening Searcher@388bba9b main
[junit4:junit4]   2> 13472 T517 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 13473 T517 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 13473 T517 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 13478 T517 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 13528 T518 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@388bba9b main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 13531 T517 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 13532 T517 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 14473 T452 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14474 T452 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:49605_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:49605"}
[junit4:junit4]   2> 14474 T452 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 14474 T452 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 14504 T451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14504 T485 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14504 T500 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14504 T470 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14504 T516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14504 T457 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14534 T517 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 14534 T517 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:49605 collection:collection1 shard:shard2
[junit4:junit4]   2> 14552 T517 oasc.ZkController.register We are http://127.0.0.1:49605/collection1/ and leader is http://127.0.0.1:50286/collection1/
[junit4:junit4]   2> 14552 T517 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49605
[junit4:junit4]   2> 14552 T517 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 14552 T517 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C32 name=collection1 org.apache.solr.core.SolrCore@6234f777 url=http://127.0.0.1:49605/collection1 node=127.0.0.1:49605_ C32_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:49605_, base_url=http://127.0.0.1:49605}
[junit4:junit4]   2> 14553 T519 C32 P49605 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 14554 T519 C32 P49605 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 14554 T517 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14554 T519 C32 P49605 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 14554 T519 C32 P49605 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 14555 T429 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 14556 T429 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 14556 T519 C32 P49605 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 14556 T429 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 14570 T429 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14572 T429 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4]   2> 14573 T429 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C31_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:40821_, base_url=http://127.0.0.1:40821}
[junit4:junit4]   2> 15277 T503 C31 P40821 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:55544/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 15283 T503 C31 P40821 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 15296 T503 C31 P40821 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:40821 START replicas=[http://127.0.0.1:55544/collection1/] nUpdates=100
[junit4:junit4]   2> 15296 T503 C31 P40821 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 15297 T503 C31 P40821 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 15297 T503 C31 P40821 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 15297 T503 C31 P40821 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 15297 T503 C31 P40821 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 15298 T503 C31 P40821 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:55544/collection1/. core=collection1
[junit4:junit4]   2> 15298 T503 C31 P40821 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C33 name=collection1 org.apache.solr.core.SolrCore@fffb761 url=http://127.0.0.1:55544/collection1 node=127.0.0.1:55544_ C33_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:55544_, base_url=http://127.0.0.1:55544, leader=true}
[junit4:junit4]   2> 15310 T461 C33 P55544 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 15324 T462 C33 P55544 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 15325 T462 C33 P55544 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty1
[junit4:junit4]   2> 15328 T462 C33 P55544 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@45782220),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 15329 T462 C33 P55544 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 15356 T462 C33 P55544 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@45782220),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@45782220),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 15356 T462 C33 P55544 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 15357 T462 C33 P55544 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty1
[junit4:junit4]   2> 15358 T462 C33 P55544 oass.SolrIndexSearcher.<init> Opening Searcher@2c93ddfb realtime
[junit4:junit4]   2> 15358 T462 C33 P55544 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15358 T462 C33 P55544 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 45
[junit4:junit4]   2> 15398 T503 C31 P40821 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 15399 T503 C31 P40821 oascsi.HttpClientUtil.createClient Creating new http client, config:connTimeout=5000&socketTimeout=20000&allowCompression=false&maxConnections=10000&maxConnectionsPerHost=10000
[junit4:junit4]   2> 15416 T503 C31 P40821 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 15419 T463 C33 P55544 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 15420 T463 C33 P55544 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 15422 T503 C31 P40821 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 15422 T503 C31 P40821 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 15422 T503 C31 P40821 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 15426 T463 C33 P55544 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty1
[junit4:junit4]   2> 15427 T463 C33 P55544 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty1/index
[junit4:junit4]   2> 15428 T463 C33 P55544 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=2 
[junit4:junit4]   2> 15429 T503 C31 P40821 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 15430 T503 C31 P40821 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3/index.20130203070712759 forceNew:false
[junit4:junit4]   2> 15430 T503 C31 P40821 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3
[junit4:junit4]   2> 15431 T503 C31 P40821 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3/index.20130203070712759 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2c17d192) fullCopy=true
[junit4:junit4]   2> 15445 T463 C33 P55544 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=8 
[junit4:junit4]   2> 15456 T503 C31 P40821 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3/index
[junit4:junit4]   2> 15456 T503 C31 P40821 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 15456 T503 C31 P40821 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130203070712759
[junit4:junit4]   2> 15458 T503 C31 P40821 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3
[junit4:junit4]   2> 15459 T503 C31 P40821 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3
[junit4:junit4]   2> 15459 T503 C31 P40821 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3/index/ new=./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3/index.20130203070712759
[junit4:junit4]   2> 15461 T503 C31 P40821 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3/index.20130203070712759 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2c17d192),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 15462 T503 C31 P40821 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 15464 T503 C31 P40821 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3
[junit4:junit4]   2> 15467 T503 C31 P40821 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 15467 T503 C31 P40821 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 15467 T503 C31 P40821 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 15468 T503 C31 P40821 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3/index.20130203070712759
[junit4:junit4]   2> 15469 T503 C31 P40821 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3
[junit4:junit4]   2> 15471 T503 C31 P40821 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3/index.20130203070712759 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2c17d192),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 15472 T503 C31 P40821 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 15472 T503 C31 P40821 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 15472 T503 C31 P40821 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3
[junit4:junit4]   2> 15472 T503 C31 P40821 oass.SolrIndexSearcher.<init> Opening Searcher@53f25552 main
[junit4:junit4]   2> 15474 T502 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@53f25552 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 15475 T502 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3/index
[junit4:junit4]   2> 15475 T503 C31 P40821 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 15477 T503 C31 P40821 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3/index.20130203070712759 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2c17d192),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3/index.20130203070712759 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2c17d192),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 15478 T503 C31 P40821 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 15479 T503 C31 P40821 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3
[junit4:junit4]   2> 15479 T503 C31 P40821 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15479 T503 C31 P40821 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3/index.20130203070712759
[junit4:junit4]   2> 15479 T503 C31 P40821 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3/index
[junit4:junit4]   2> 15479 T503 C31 P40821 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 15480 T503 C31 P40821 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 15480 T503 C31 P40821 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 15480 T503 C31 P40821 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 15488 T503 C31 P40821 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 15575 T429 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16033 T452 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16034 T452 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:49605_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:49605"}
[junit4:junit4]   2> 16041 T452 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:40821_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:40821"}
[junit4:junit4]   2> 16061 T451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16061 T457 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16061 T516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16061 T470 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16061 T500 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16061 T485 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16577 T429 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17582 T429 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C32_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:49605_, base_url=http://127.0.0.1:49605}
[junit4:junit4]   2> 18576 T519 C32 P49605 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:50286/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 18576 T519 C32 P49605 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:49605 START replicas=[http://127.0.0.1:50286/collection1/] nUpdates=100
[junit4:junit4]   2> 18576 T519 C32 P49605 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 18577 T519 C32 P49605 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 18577 T519 C32 P49605 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 18577 T519 C32 P49605 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 18577 T519 C32 P49605 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 18577 T519 C32 P49605 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:50286/collection1/. core=collection1
[junit4:junit4]   2> 18577 T519 C32 P49605 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 18584 T429 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> ASYNC  NEW_CORE C34 name=collection1 org.apache.solr.core.SolrCore@1930f2a url=http://127.0.0.1:50286/collection1 node=127.0.0.1:50286_ C34_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:50286_, base_url=http://127.0.0.1:50286, leader=true}
[junit4:junit4]   2> 18589 T476 C34 P50286 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 18594 T477 C34 P50286 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 18595 T477 C34 P50286 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty2
[junit4:junit4]   2> 18598 T477 C34 P50286 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@644151dc),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 18598 T477 C34 P50286 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 18601 T477 C34 P50286 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@644151dc),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@644151dc),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 18601 T477 C34 P50286 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 18602 T477 C34 P50286 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty2
[junit4:junit4]   2> 18603 T477 C34 P50286 oass.SolrIndexSearcher.<init> Opening Searcher@691a5672 realtime
[junit4:junit4]   2> 18603 T477 C34 P50286 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 18603 T477 C34 P50286 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 9
[junit4:junit4]   2> 18604 T519 C32 P49605 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 18604 T519 C32 P49605 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 18606 T478 C34 P50286 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 18607 T478 C34 P50286 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 18607 T519 C32 P49605 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 18607 T519 C32 P49605 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 18607 T519 C32 P49605 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 18610 T478 C34 P50286 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty2
[junit4:junit4]   2> 18611 T478 C34 P50286 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty2/index
[junit4:junit4]   2> 18611 T478 C34 P50286 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1 
[junit4:junit4]   2> 18612 T519 C32 P49605 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 18612 T519 C32 P49605 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4/index.20130203070715942 forceNew:false
[junit4:junit4]   2> 18613 T519 C32 P49605 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4
[junit4:junit4]   2> 18613 T519 C32 P49605 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4/index.20130203070715942 lockFactory=org.apache.lucene.store.NativeFSLockFactory@185d4ea7) fullCopy=true
[junit4:junit4]   2> 18616 T478 C34 P50286 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> 18618 T519 C32 P49605 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4/index
[junit4:junit4]   2> 18618 T519 C32 P49605 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 18619 T519 C32 P49605 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130203070715942
[junit4:junit4]   2> 18620 T519 C32 P49605 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4
[junit4:junit4]   2> 18620 T519 C32 P49605 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4
[junit4:junit4]   2> 18621 T519 C32 P49605 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4/index/ new=./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4/index.20130203070715942
[junit4:junit4]   2> 18623 T519 C32 P49605 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4/index.20130203070715942 lockFactory=org.apache.lucene.store.NativeFSLockFactory@185d4ea7),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 18623 T519 C32 P49605 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 18625 T519 C32 P49605 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4
[junit4:junit4]   2> 18625 T519 C32 P49605 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 18625 T519 C32 P49605 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 18625 T519 C32 P49605 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 18626 T519 C32 P49605 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4/index.20130203070715942
[junit4:junit4]   2> 18626 T519 C32 P49605 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4
[junit4:junit4]   2> 18629 T519 C32 P49605 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4/index.20130203070715942 lockFactory=org.apache.lucene.store.NativeFSLockFactory@185d4ea7),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 18629 T519 C32 P49605 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 18629 T519 C32 P49605 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 18630 T519 C32 P49605 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4
[junit4:junit4]   2> 18630 T519 C32 P49605 oass.SolrIndexSearcher.<init> Opening Searcher@44c36d6d main
[junit4:junit4]   2> 18631 T518 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@44c36d6d main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 18632 T518 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4/index
[junit4:junit4]   2> 18632 T519 C32 P49605 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 18635 T519 C32 P49605 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4/index.20130203070715942 lockFactory=org.apache.lucene.store.NativeFSLockFactory@185d4ea7),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4/index.20130203070715942 lockFactory=org.apache.lucene.store.NativeFSLockFactory@185d4ea7),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 18635 T519 C32 P49605 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 18636 T519 C32 P49605 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4
[junit4:junit4]   2> 18636 T519 C32 P49605 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 18636 T519 C32 P49605 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4/index.20130203070715942
[junit4:junit4]   2> 18636 T519 C32 P49605 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4/index
[junit4:junit4]   2> 18637 T519 C32 P49605 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 18637 T519 C32 P49605 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 18637 T519 C32 P49605 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 18637 T519 C32 P49605 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 18645 T519 C32 P49605 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 19070 T452 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19071 T452 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:49605_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:49605"}
[junit4:junit4]   2> 19087 T451 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19088 T470 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19088 T457 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19088 T500 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19088 T485 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19088 T516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19586 T429 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19588 T429 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4]   2> ASYNC  NEW_CORE C35 name=collection1 org.apache.solr.core.SolrCore@60f1108e url=http://127.0.0.1:38869/collection1 node=127.0.0.1:38869_ C35_STATE=coll:control_collection core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:38869_, base_url=http://127.0.0.1:38869, leader=true}
[junit4:junit4]   2> 19602 T441 C35 P38869 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/control/data
[junit4:junit4]   2> 19606 T441 C35 P38869 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2385857),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 19606 T441 C35 P38869 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 19607 T441 C35 P38869 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/control/data
[junit4:junit4]   2> 19608 T441 C35 P38869 oass.SolrIndexSearcher.<init> Opening Searcher@299eb4d0 realtime
[junit4:junit4]   2> 19608 T441 C35 P38869 /update {wt=javabin&version=2} {deleteByQuery=*:* (-1425940086188408832)} 0 10
[junit4:junit4]   2> 19617 T479 C34 P50286 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty2
[junit4:junit4]   2> 19633 T464 C33 P55544 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty1
[junit4:junit4]   2>  C32_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:49605_, base_url=http://127.0.0.1:49605}
[junit4:junit4]   2> 19638 T506 C32 P49605 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty4
[junit4:junit4]   2> 19639 T506 C32 P49605 /update {update.distrib=FROMLEADER&_version_=-1425940086203088896&update.from=http://127.0.0.1:50286/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1425940086203088896)} 0 2
[junit4:junit4]   2>  C31_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:40821_, base_url=http://127.0.0.1:40821}
[junit4:junit4]   2> 19643 T490 C31 P40821 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359882417335/jetty3
[junit4:junit4]   2> 19643 T490 C31 P40821 /update {update.distrib=FROMLEADER&_version_=-1425940086219866112&update.from=http://127.0.0.1:55544/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1425940086219866112)} 0 2
[junit4:junit4]   2> 19644 T464 C33 P55544 /update {update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1425940086219866112)} 0 13
[junit4:junit4]   2> 19645 T479 C34 P50286 /update {wt=javabin&version=2} {deleteByQuery=*:* (-1425940086203088896)} 0 34
[junit4:junit4]   2> 19654 T441 C35 P38869 /update {wt=javabin&version=2} {add=[1 (1425940086239789056)]} 0 4
[junit4:junit4]   2> 19671 T490 C31 P40821 /update {distrib.from=http://127.0.0.1:55544/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1425940086246080512)]} 0 3
[junit4:junit4]   2> 19672 T465 C33 P55544 /update {wt=javabin&version=2} {add=[1 (1425940086246080512)]} 0 16
[junit4:junit4]   2> 19677 T441 C35 P38869 /update {wt=javabin&version=2} {add=[2 (1425940086266003456)]} 0 2
[junit4:junit4]   2> 19693 T506 C32 P49605 /update {distrib.from=http://127.0.0.1:50286/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1425940086275440640)]} 0 3
[junit4:junit4]   2> 19694 T480 C34 P50286 /update {distrib.from=http://127.0.0.1:55544/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[2 (1425940086275440640)]} 0 10
[junit4:junit4]   2> 19694 T465 C33 P55544 /update {wt=javabin&version=2} {add=[2]} 0 15
[junit4:junit4]   2> 19699 T441 C35 P38869 /update {wt=javabin&version=2} {add=[3 (1425940086289072128)]} 0 2
[junit4:junit4]   2> 19706 T506 C32 P49605 /update {distrib.from=http://127.0.0.1:50286/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[3 (1425940086292217856)]} 0 1
[junit4:junit4]   2> 19707 T479 C34 P50286 /update {wt=javabin&version=2} {add=[3 (1425940086292217856)]} 0 6
[junit4:junit4]   2> 19712 T441 C35 P38869 /update {wt=javabin&version=2} {add=[4 (1425940086301655040)]} 0 2
[junit4:junit4]   2> 19719 T490 C31 P40821 /update {distrib.from=http://127.0.0.1:55544/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[4 (1425940086305849344)]} 0 1
[junit4:junit4]   2> 19720 T465 C33 P55544 /update {wt=javabin&version=2} {add=[4 (1425940086305849344)]} 0 6
[junit4:junit4]   2> 19726 T441 C35 P38869 /update {wt=javabin&version=2} {add=[5 (1425940086316335104)]} 0 2
[junit4:junit4]   2> 19736 T506 C32 P49605 /update {distrib.from=http://127.0.0.1:50286/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[5 (1425940086323675136)]} 0 1
[junit4:junit4]   2> 19737 T480 C34 P50286 /update {distrib.from=http://127.0.0.1:55544/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[5 (1425940086323675136)]} 0 6
[junit4:junit4]   2> 19738 T465 C33 P55544 /update {wt=javabin&version=2} {add=[5]} 0 10
[junit4:junit4]   2> 19742 T441 C35 P38869 /update {wt=javabin&version=2} {add=[6 (1425940086333112320)]} 0 2
[junit4:junit4]   2> 19749 T506 C32 P49605 /update {distrib.from=http://127.0.0.1:50286/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[6 (1425940086337306624)]} 0 1
[junit4:junit4]   2> 19750 T479 C34 P50286 /update {wt=javabin&version=2} {add=[6 (1425940086337306624)]} 0 6
[junit4:junit4]   2> 19755 T441 C35 P38869 /update {wt=javabin&version=2} {add=[7 (1425940086346743808)]} 0 2
[junit4:junit4]   2> 19762 T506 C32 P49605 /update {distrib.from=http://127.0.0.1:50286/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[7 (1425940086350938112)]} 0 1
[junit4:junit4]   2> 19763 T479 C34 P50286 /update {wt=javabin&version=2} {add=[7 (1425940086350938112)]} 0 6
[junit4:junit4]   2> 19768 T441 C35 P38869 /update {wt=javabin&version=2} {add=[8 (1425940086361423872)]} 0 2
[junit4:junit4]   2> 19776 T490 C31 P40821 /update {distrib.from=http://127.0.0.1:55544/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[8 (1425940086365618176)]} 0 1
[junit4:junit4]   2> 19777 T465 C33 P55544 /update {wt=javabin&version=2} {add=[8 (1425940086365618176)]} 0 6
[junit4:junit4]   2> 19782 T441 C35 P38869 /update {wt=javabin&version=2} {add=[9 (1425940086375055360)]} 0 2
[junit4:junit4]   2> 19789 T506 C32 P49605 /update {distrib.from=http://127.0.0.1:50286/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[9 (1425940086379249664)]} 0 1
[junit4:junit4]   2> 19790 T479 C34 P50286 /update {wt=javabin&version=2} {add=[9 (1425940086379249664)]} 0 6
[junit4:junit4]   2> 19835 T441 C35 P38869 /update {wt=javabin&version=2} {add=[10 (1425940086388686848)]} 0 42
[junit4:junit4]   2> 19844 T490 C31 P40821 /update {distrib.from=http://127.0.0.1:55544/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10 (1425940086437969920)]} 0 1
[junit4:junit4]   2> 19845 T465 C33 P55544 /update {wt=javabin&version=2} {add=[10 (1425940086437969920)]} 0 6
[junit4:junit4]   2> 19851 T441 C35 P38869 /update {wt=javabin&version=2} {add=[11 (1425940086447407104)]} 0 3
[junit4:junit4]   2> 19862 T490 C31 P40821 /update {distrib.from=http://127.0.0.1:55544/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[11 (1425940086455795712)]} 0 1
[junit4:junit4]   2> 19863 T464 C33 P55544 /update {distrib.from=http://127.0.0.1:50286/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[11 (1425940086455795712)]} 0 6
[junit4:junit4]   2> 19863 T479 C34 P50286 /update {wt=javabin&version=2} {add=[11]} 0 9
[junit4:junit4]   2> 19869 T441 C35 P38869 /update {wt=javabin&version=2} {add=[12 (1425940086467330048)]} 0 2
[junit4:junit4]   2> 19880 T490 C31 P40821 /update {distrib.from=http://127.0.0.1:55544/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[12 (1425940086474670080)]} 0 1
[junit4:junit4]   2> 19881 T464 C33 P55544 /update {distrib.from=http://127.0.0.1:50286/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[12 (1425940086474670080)]} 0 6
[junit4:junit4]   2> 19882 T479 C34 P50286 /update {wt=javabin&version=2} {add=[12]} 0 10
[junit4:junit4]   2> 19887 T441 C35 P38869 /update {wt=javabin&version=2} {add=[13 (1425940086486204416)]} 0 2
[junit4:junit4]   2> 19899 T490 C31 P40821 /update {distrib.from=http://127.0.0.1:55544/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[13 (1425940086493544448)]} 0 2
[junit4:junit4]   2> 19899 T464 C33 P55544 /update {distrib.from=http://127.0.0.1:50286/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[13 (1425940086493544448)]} 0 6
[junit4:junit4]   2> 19900 T479 C34 P50286 /update {wt=javabin&version=2} {add=[13]} 0 10
[junit4:junit4]   2> 19905 T441 C35 P38869 /update {wt=javabin&version=2} {add=[14 (1425940086505078784)]} 0 2
[junit4:junit4]   2> 19916 T490 C31 P40821 /update {distrib.from=http://127.0.0.1:55544/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[14 (1425940086512418816)]} 0 1
[junit4:junit4]   2> 19917 T464 C33 P55544 /update {distrib.from=http://127.0.0.1:50286/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[14 (1425940086512418816)]} 0 6
[junit4:junit4]   2> 19917 T479 C34 P50286 /update {wt=javabin&version=2} {add=[14]} 0 9
[junit4:junit4]   2> 19922 T441 C35 P38869 /update {wt=javabin&version=2} {add=[15 (1425940086522904576)]} 0 2
[junit4:junit4]   2> 19933 T490 C31 P40821 /update {distrib.from=http://127.0.0.1:55544/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[15 (1425940086531293184)]} 0 1
[junit4:junit4]   2> 19934 T464 C33 P55544 /update {distrib.from=http://127.0.0.1:50286/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[15 (1425940086531293184)]} 0 6
[junit4:junit4]   2> 19934 T479 C34 P50286 /update {wt=javabin&version=2} {add=[15]} 0 9
[junit4:junit4]   2> 19940 T441 C35 P38869 /update {wt=javabin&version=2} {add=[16 (1425940086540730368)]} 0 3
[junit4:junit4]   2> 20010 T490 C31 P40821 /update {distrib.from=http://127.0.0.1:55544/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[16 (1425940086550167552)]} 0 29
[junit4:junit4]   2> 20011 T464 C33 P55544 /update {distrib.from=http://127.0.0.1:50286/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[16 (1425940086550167552)]} 0 65
[junit4:junit4]   2> 20012 T479 C34 P50286 /update {wt=javabin&version=2} {add=[16]} 0 69
[junit4:junit4]   2> 20023 T441 C35 P38869 /update {wt=javabin&version=2} {add=[17 (1425940086624616448)]} 0

[...truncated too long message...]

te=TIMED_WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at java.lang.Object.wait(Native Method)
[junit4:junit4]   2> 	        at java.util.TimerThread.mainLoop(Timer.java:552)
[junit4:junit4]   2> 	        at java.util.TimerThread.run(Timer.java:505)
[junit4:junit4]   2> 	  39) Thread[id=450, name=TEST-BasicDistributedZkTest.testDistribSearch-seed#[B1AAA651BCD0851A]-SendThread(localhost:18580), state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.nio.ch.KQueueArrayWrapper.interrupt(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.KQueueArrayWrapper.interrupt(KQueueArrayWrapper.java:164)
[junit4:junit4]   2> 	        at sun.nio.ch.KQueueSelectorImpl.wakeup(KQueueSelectorImpl.java:245)
[junit4:junit4]   2> 	        at java.nio.channels.spi.AbstractSelector$1.interrupt(AbstractSelector.java:210)
[junit4:junit4]   2> 	        at java.nio.channels.spi.AbstractSelector.begin(AbstractSelector.java:216)
[junit4:junit4]   2> 	        at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:102)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
[junit4:junit4]   2> 	  40) Thread[id=657, name=searcherExecutor-386-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  41) Thread[id=502, name=searcherExecutor-270-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  42) Thread[id=851, name=searcherExecutor-412-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  43) Thread[id=576, name=searcherExecutor-324-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> NOTE: test params are: codec=Lucene42: {n_f1=PostingsFormat(name=Memory doPackFST= true), foo_b=MockVariableIntBlock(baseBlockSize=37), cat=MockFixedIntBlock(blockSize=758), foo_d=PostingsFormat(name=Lucene41WithOrds), foo_f=MockVariableIntBlock(baseBlockSize=37), n_tl1=PostingsFormat(name=Memory doPackFST= true), n_d1=MockFixedIntBlock(blockSize=758), rnd_b=PostingsFormat(name=Lucene41WithOrds), intDefault=PostingsFormat(name=Memory doPackFST= true), n_td1=PostingsFormat(name=Memory doPackFST= true), timestamp=MockFixedIntBlock(blockSize=758), id=PostingsFormat(name=Lucene41WithOrds), range_facet_sl=PostingsFormat(name=Lucene41WithOrds), range_facet_si=MockFixedIntBlock(blockSize=758), oddField_s=PostingsFormat(name=Lucene41WithOrds), sequence_i=PostingsFormat(name=Lucene41WithOrds), name=PostingsFormat(name=Lucene41WithOrds), foo_i=PostingsFormat(name=Memory doPackFST= true), regex_dup_B_s=MockFixedIntBlock(blockSize=758), multiDefault=MockFixedIntBlock(blockSize=758), n_tf1=MockFixedIntBlock(blockSize=758), n_dt1=PostingsFormat(name=Memory doPackFST= true), genre_s=PostingsFormat(name=Lucene41WithOrds), author_t=PostingsFormat(name=Memory doPackFST= true), n_ti1=PostingsFormat(name=Lucene41WithOrds), range_facet_l=MockFixedIntBlock(blockSize=758), text=MockVariableIntBlock(baseBlockSize=37), _version_=PostingsFormat(name=Memory doPackFST= true), val_i=PostingsFormat(name=Lucene41WithOrds), SubjectTerms_mfacet=PostingsFormat(name=Memory doPackFST= true), series_t=PostingsFormat(name=Memory doPackFST= true), a_t=MockFixedIntBlock(blockSize=758), n_tdt1=PostingsFormat(name=Memory doPackFST= true), regex_dup_A_s=MockVariableIntBlock(baseBlockSize=37), price=MockVariableIntBlock(baseBlockSize=37), other_tl1=PostingsFormat(name=Memory doPackFST= true), n_l1=MockFixedIntBlock(blockSize=758), a_si=PostingsFormat(name=Memory doPackFST= true), inStock=MockVariableIntBlock(baseBlockSize=37)}, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=tr, timezone=America/Bahia
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=46,free=259081800,total=377487360
[junit4:junit4]   2> NOTE: All tests run in this JVM: [ClusterStateUpdateTest, TestComponentsName, QueryElevationComponentTest, ResourceLoaderTest, SOLR749Test, TestSolrQueryParser, DistributedSpellCheckComponentTest, SpatialFilterTest, FileBasedSpellCheckerTest, TestPerFieldSimilarity, TestSolrIndexConfig, TestStressVersions, TermsComponentTest, CopyFieldTest, TestXIncludeConfig, SimpleFacetsTest, TestFaceting, JSONWriterTest, ChaosMonkeySafeLeaderTest, DirectUpdateHandlerOptimizeTest, TestWordDelimiterFilterFactory, BadComponentTest, TestBadConfig, ExternalFileFieldSortTest, FastVectorHighlighterTest, TestPHPSerializedResponseWriter, TestJoin, CircularListTest, TestFuzzyAnalyzedSuggestions, TestZkChroot, TestSystemIdResolver, BasicDistributedZkTest]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=BasicDistributedZkTest -Dtests.seed=B1AAA651BCD0851A -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=tr -Dtests.timezone=America/Bahia -Dtests.file.encoding=UTF-8
[junit4:junit4] ERROR   0.00s J0 | BasicDistributedZkTest (suite) <<<
[junit4:junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([B1AAA651BCD0851A]:0)
[junit4:junit4] Completed on J0 in 7224.12s, 1 test, 2 errors <<< FAILURES!

[...truncated 98 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:346: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/common-build.xml:428: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:1176: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:841: There were test failures: 249 suites, 1038 tests, 1 suite-level error, 1 error, 128 ignored (4 assumptions)

Total time: 145 minutes 8 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-trunk-java7 - Build # 3703 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-java7/3703/

3 tests failed.
REGRESSION:  org.apache.solr.cloud.BasicDistributedZkTest.testDistribSearch

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([68A457610324D72F]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([68A457610324D72F]:0)


FAILED:  org.apache.solr.cloud.SyncSliceTest.testDistribSearch

Error Message:
shard1 should have just been set up to be inconsistent - but it's still consistent

Stack Trace:
java.lang.AssertionError: shard1 should have just been set up to be inconsistent - but it's still consistent
	at __randomizedtesting.SeedInfo.seed([68A457610324D72F:E942D979747BB713]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNotNull(Assert.java:526)
	at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:214)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:794)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:722)




Build Log:
[...truncated 8820 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.SyncSliceTest
[junit4:junit4]   2> 1 T68 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /_pq/
[junit4:junit4]   2> 6 T68 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./solrtest-SyncSliceTest-1359843844230
[junit4:junit4]   2> 13 T68 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 19 T69 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 91 T69 oaz.Environment.logEnv Server environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT
[junit4:junit4]   2> 92 T69 oaz.Environment.logEnv Server environment:host.name=lucene.zones.apache.org
[junit4:junit4]   2> 92 T69 oaz.Environment.logEnv Server environment:java.version=1.7.0_09
[junit4:junit4]   2> 92 T69 oaz.Environment.logEnv Server environment:java.vendor=Oracle Corporation
[junit4:junit4]   2> 92 T69 oaz.Environment.logEnv Server environment:java.home=/usr/local/openjdk7/jre
[junit4:junit4]   2> 92 T69 oaz.Environment.logEnv Server environment:java.class.path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/classes/test:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/codecs/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-solrj/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/analysis/common/lucene-analyzers-common-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/highlighter/lucene-highlighter-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/memory/lucene-memory-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/misc/lucene-misc-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/spatial/lucene-spatial-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/suggest/lucene-suggest-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/grouping/lucene-grouping-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/queries/lucene-queries-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/queryparser/lucene-queryparser-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/cglib-nodep-2.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/commons-cli-1.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/commons-codec-1.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/commons-fileupload-1.2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/commons-lang-2.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/easymock-3.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/guava-13.0.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/javax.servlet-api-3.0.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/objenesis-1.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/spatial4j-0.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/commons-io-2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/httpclient-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/httpcore-4.1.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/httpmime-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/jcl-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/slf4j-api-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/slf4j-jdk14-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/wstx-asl-3.2.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/zookeeper-3.4.5.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-continuation-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-deploy-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-http-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-io-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-jmx-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-security-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-server-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-servlet-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-util-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-webapp-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-xml-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/servlet-api-3.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/example-DIH/solr/db/lib/derby-10.9.1.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/test-framework/lib/junit-4.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/test-framework/lib/randomizedtesting-runner-2.0.8.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/home/hudson/.ant/lib/ivy-2.2.0.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/usr/local/openjdk7/lib/tools.jar:/home/hudson/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.0.8.jar
[junit4:junit4]   2> 94 T69 oaz.Environment.logEnv Server environment:java.library.path=/usr/local/openjdk7/jre/lib/amd64/server:/usr/local/openjdk7/jre/lib/amd64:/usr/local/openjdk7/jre/../lib/amd64:/usr/local/openjdk6/jre/lib/amd64/server:/usr/local/openjdk6/jre/lib/amd64:/usr/local/openjdk6/jre/../lib/amd64:/usr/java/packages/lib/amd64:/lib:/usr/lib:/usr/local/lib
[junit4:junit4]   2> 94 T69 oaz.Environment.logEnv Server environment:java.io.tmpdir=.
[junit4:junit4]   2> 94 T69 oaz.Environment.logEnv Server environment:java.compiler=<NA>
[junit4:junit4]   2> 94 T69 oaz.Environment.logEnv Server environment:os.name=FreeBSD
[junit4:junit4]   2> 95 T69 oaz.Environment.logEnv Server environment:os.arch=amd64
[junit4:junit4]   2> 95 T69 oaz.Environment.logEnv Server environment:os.version=9.0-RELEASE
[junit4:junit4]   2> 95 T69 oaz.Environment.logEnv Server environment:user.name=hudson
[junit4:junit4]   2> 95 T69 oaz.Environment.logEnv Server environment:user.home=/home/hudson
[junit4:junit4]   2> 95 T69 oaz.Environment.logEnv Server environment:user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 139 T69 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 140 T69 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4]   2> 140 T69 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4]   2> 146 T69 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 219 T69 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4]   2> 220 T68 oasc.ZkTestServer.run start zk server on port:16522
[junit4:junit4]   2> 290 T68 oaz.Environment.logEnv Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT
[junit4:junit4]   2> 291 T68 oaz.Environment.logEnv Client environment:host.name=lucene.zones.apache.org
[junit4:junit4]   2> 291 T68 oaz.Environment.logEnv Client environment:java.version=1.7.0_09
[junit4:junit4]   2> 292 T68 oaz.Environment.logEnv Client environment:java.vendor=Oracle Corporation
[junit4:junit4]   2> 292 T68 oaz.Environment.logEnv Client environment:java.home=/usr/local/openjdk7/jre
[junit4:junit4]   2> 292 T68 oaz.Environment.logEnv Client environment:java.class.path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/classes/test:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/test-framework/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/codecs/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-solrj/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/analysis/common/lucene-analyzers-common-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/highlighter/lucene-highlighter-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/memory/lucene-memory-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/misc/lucene-misc-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/spatial/lucene-spatial-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/suggest/lucene-suggest-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/grouping/lucene-grouping-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/queries/lucene-queries-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/queryparser/lucene-queryparser-5.0-SNAPSHOT.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/cglib-nodep-2.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/commons-cli-1.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/commons-codec-1.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/commons-fileupload-1.2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/commons-lang-2.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/easymock-3.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/guava-13.0.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/javax.servlet-api-3.0.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/objenesis-1.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/core/lib/spatial4j-0.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/commons-io-2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/httpclient-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/httpcore-4.1.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/httpmime-4.1.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/jcl-over-slf4j-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/slf4j-api-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/slf4j-jdk14-1.6.4.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/wstx-asl-3.2.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/solrj/lib/zookeeper-3.4.5.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-continuation-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-deploy-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-http-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-io-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-jmx-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-security-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-server-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-servlet-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-util-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-webapp-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/jetty-xml-8.1.8.v20121106.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/lib/servlet-api-3.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/example-DIH/solr/db/lib/derby-10.9.1.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/test-framework/lib/junit-4.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/test-framework/lib/randomizedtesting-runner-2.0.8.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/home/hudson/.ant/lib/ivy-2.2.0.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/usr/local/openjdk7/lib/tools.jar:/home/hudson/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.0.8.jar
[junit4:junit4]   2> 294 T68 oaz.Environment.logEnv Client environment:java.library.path=/usr/local/openjdk7/jre/lib/amd64/server:/usr/local/openjdk7/jre/lib/amd64:/usr/local/openjdk7/jre/../lib/amd64:/usr/local/openjdk6/jre/lib/amd64/server:/usr/local/openjdk6/jre/lib/amd64:/usr/local/openjdk6/jre/../lib/amd64:/usr/java/packages/lib/amd64:/lib:/usr/lib:/usr/local/lib
[junit4:junit4]   2> 294 T68 oaz.Environment.logEnv Client environment:java.io.tmpdir=.
[junit4:junit4]   2> 294 T68 oaz.Environment.logEnv Client environment:java.compiler=<NA>
[junit4:junit4]   2> 295 T68 oaz.Environment.logEnv Client environment:os.name=FreeBSD
[junit4:junit4]   2> 295 T68 oaz.Environment.logEnv Client environment:os.arch=amd64
[junit4:junit4]   2> 296 T68 oaz.Environment.logEnv Client environment:os.version=9.0-RELEASE
[junit4:junit4]   2> 296 T68 oaz.Environment.logEnv Client environment:user.name=hudson
[junit4:junit4]   2> 296 T68 oaz.Environment.logEnv Client environment:user.home=/home/hudson
[junit4:junit4]   2> 297 T68 oaz.Environment.logEnv Client environment:user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 301 T68 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:16522 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@5cccc9d8
[junit4:junit4]   2> 366 T68 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 376 T74 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:16522. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 379 T74 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:16522, initiating session
[junit4:junit4]   2> 380 T70 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:35339
[junit4:junit4]   2> 412 T70 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:35339
[junit4:junit4]   2> 428 T72 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 462 T72 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9d02617d0000 with negotiated timeout 10000 for client /140.211.11.196:35339
[junit4:junit4]   2> 462 T74 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:16522, sessionid = 0x13c9d02617d0000, negotiated timeout = 10000
[junit4:junit4]   2> 468 T75 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5cccc9d8 name:ZooKeeperConnection Watcher:127.0.0.1:16522 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 469 T68 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 470 T68 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 530 T73 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c9d02617d0000
[junit4:junit4]   2> 531 T75 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 531 T68 oaz.ZooKeeper.close Session: 0x13c9d02617d0000 closed
[junit4:junit4]   2> 532 T70 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:35339 which had sessionid 0x13c9d02617d0000
[junit4:junit4]   2> 532 T68 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:16522/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@f2d5171
[junit4:junit4]   2> 533 T68 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 533 T76 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:16522. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 534 T76 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:16522, initiating session
[junit4:junit4]   2> 534 T70 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:44969
[junit4:junit4]   2> 535 T70 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:44969
[junit4:junit4]   2> 543 T72 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9d02617d0001 with negotiated timeout 10000 for client /140.211.11.196:44969
[junit4:junit4]   2> 543 T76 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:16522, sessionid = 0x13c9d02617d0001, negotiated timeout = 10000
[junit4:junit4]   2> 544 T77 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f2d5171 name:ZooKeeperConnection Watcher:127.0.0.1:16522/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 544 T68 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 544 T68 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 550 T68 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 555 T68 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 558 T68 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 565 T68 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 588 T68 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 602 T68 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 611 T68 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 717 T68 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 718 T68 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 723 T68 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 723 T68 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 728 T68 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 728 T68 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 732 T68 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 733 T68 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 737 T68 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 738 T68 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 743 T68 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 743 T68 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 747 T68 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 748 T68 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 752 T73 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c9d02617d0001
[junit4:junit4]   2> 753 T77 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 753 T70 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:44969 which had sessionid 0x13c9d02617d0001
[junit4:junit4]   2> 753 T68 oaz.ZooKeeper.close Session: 0x13c9d02617d0001 closed
[junit4:junit4]   2> 893 T68 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 897 T68 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:15097
[junit4:junit4]   2> 897 T68 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 898 T68 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 898 T68 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1359843844978
[junit4:junit4]   2> 899 T68 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1359843844978/solr.xml
[junit4:junit4]   2> 899 T68 oasc.CoreContainer.<init> New CoreContainer 196684649
[junit4:junit4]   2> 900 T68 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1359843844978/'
[junit4:junit4]   2> 900 T68 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1359843844978/'
[junit4:junit4]   2> 960 T68 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 960 T68 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 961 T68 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 962 T68 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 962 T68 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 963 T68 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 963 T68 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 964 T68 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 964 T68 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 965 T68 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 994 T68 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 1021 T68 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:16522/solr
[junit4:junit4]   2> 1056 T68 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 1057 T68 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:16522 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@71f1ce16
[junit4:junit4]   2> 1058 T68 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1059 T87 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:16522. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 1059 T87 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:16522, initiating session
[junit4:junit4]   2> 1060 T70 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:15129
[junit4:junit4]   2> 1060 T70 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:15129
[junit4:junit4]   2> 1062 T72 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9d02617d0002 with negotiated timeout 20000 for client /140.211.11.196:15129
[junit4:junit4]   2> 1062 T87 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:16522, sessionid = 0x13c9d02617d0002, negotiated timeout = 20000
[junit4:junit4]   2> 1063 T88 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@71f1ce16 name:ZooKeeperConnection Watcher:127.0.0.1:16522 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1063 T68 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1065 T73 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c9d02617d0002
[junit4:junit4]   2> 1067 T88 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 1067 T70 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:15129 which had sessionid 0x13c9d02617d0002
[junit4:junit4]   2> 1067 T68 oaz.ZooKeeper.close Session: 0x13c9d02617d0002 closed
[junit4:junit4]   2> 1072 T68 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 1092 T68 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:16522/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@3add6825
[junit4:junit4]   2> 1093 T68 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 1094 T89 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:16522. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 1095 T89 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:16522, initiating session
[junit4:junit4]   2> 1095 T70 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:11596
[junit4:junit4]   2> 1095 T70 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:11596
[junit4:junit4]   2> 1097 T72 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9d02617d0003 with negotiated timeout 20000 for client /140.211.11.196:11596
[junit4:junit4]   2> 1097 T89 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:16522, sessionid = 0x13c9d02617d0003, negotiated timeout = 20000
[junit4:junit4]   2> 1098 T90 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3add6825 name:ZooKeeperConnection Watcher:127.0.0.1:16522/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 1099 T68 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 1117 T73 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9d02617d0003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 1130 T68 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 1133 T68 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:15097__pq
[junit4:junit4]   2> 1141 T73 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9d02617d0003 type:delete cxid:0x7 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:15097__pq Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:15097__pq
[junit4:junit4]   2> 1142 T68 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:15097__pq
[junit4:junit4]   2> 1148 T68 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 1179 T73 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9d02617d0003 type:delete cxid:0x15 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 1181 T68 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 1184 T68 oasc.Overseer.start Overseer (id=89118726191644675-127.0.0.1:15097__pq-n_0000000000) starting
[junit4:junit4]   2> 1185 T73 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9d02617d0003 type:create cxid:0x1a zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 1212 T73 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9d02617d0003 type:create cxid:0x1b zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 1214 T73 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9d02617d0003 type:create cxid:0x1c zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 1216 T73 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9d02617d0003 type:create cxid:0x1d zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 1224 T92 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 1225 T68 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 1229 T68 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 1250 T91 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 1264 T93 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1359843844978/collection1
[junit4:junit4]   2> 1265 T93 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 1266 T93 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 1266 T93 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 1268 T93 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1359843844978/collection1/'
[junit4:junit4]   2> 1270 T93 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-controljetty-1359843844978/collection1/lib/README' to classloader
[junit4:junit4]   2> 1270 T93 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-controljetty-1359843844978/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 1345 T93 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 1410 T93 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1512 T93 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 1530 T93 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 2106 T93 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 2119 T93 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 2122 T93 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 2152 T93 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2158 T93 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 2163 T93 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2167 T93 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 2170 T93 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1359843844978/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1359843844230/control/data/
[junit4:junit4]   2> 2174 T93 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@71ac4afb
[junit4:junit4]   2> 2175 T93 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 2201 T93 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/control/data forceNew:false
[junit4:junit4]   2> 2202 T93 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/control/data
[junit4:junit4]   2> 2202 T93 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1359843844230/control/data/index/
[junit4:junit4]   2> 2202 T93 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1359843844230/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 2203 T93 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/control/data/index forceNew:false
[junit4:junit4]   2> 2237 T93 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359843844230/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3a56ad08; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 2238 T93 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 2239 T93 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/control/data/index
[junit4:junit4]   2> 2239 T93 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 2242 T93 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 2243 T93 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 2244 T93 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 2244 T93 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 2245 T93 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 2246 T93 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 2246 T93 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 2247 T93 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 2248 T93 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 2248 T93 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 2249 T93 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 2250 T93 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 2250 T93 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 2250 T93 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 2251 T93 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 2251 T93 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 2252 T93 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2252 T93 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2253 T93 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2253 T93 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2254 T93 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2254 T93 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2255 T93 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2255 T93 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 2256 T93 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 2256 T93 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 2266 T93 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 2269 T93 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 2271 T93 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 2273 T93 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/control/data
[junit4:junit4]   2> 2275 T93 oass.SolrIndexSearcher.<init> Opening Searcher@13f0d199 main
[junit4:junit4]   2> 2293 T93 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 2294 T93 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 2294 T93 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 2301 T93 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 2351 T94 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@13f0d199 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 2356 T93 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 2356 T93 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 2357 T73 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9d02617d0003 type:create cxid:0x50 zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 2756 T91 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2758 T91 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:15097__pq",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:15097/_pq"}
[junit4:junit4]   2> 2758 T91 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4]   2> 2778 T91 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 2779 T73 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9d02617d0003 type:create cxid:0x58 zxid:0x2c txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 2822 T90 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 3371 T93 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 3371 T93 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:15097/_pq collection:control_collection shard:shard1
[junit4:junit4]   2> 3379 T93 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 3393 T93 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4]   2> 3412 T73 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9d02617d0003 type:delete cxid:0x72 zxid:0x36 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4]   2> 3414 T93 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 3415 T73 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9d02617d0003 type:create cxid:0x73 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3424 T93 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 3424 T93 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 3431 T93 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:15097/_pq/collection1/
[junit4:junit4]   2> 3432 T93 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 3432 T93 oasc.SyncStrategy.syncToMe http://127.0.0.1:15097/_pq/collection1/ has no replicas
[junit4:junit4]   2> 3433 T93 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:15097/_pq/collection1/
[junit4:junit4]   2> 3433 T93 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4]   2> 3441 T73 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9d02617d0003 type:create cxid:0x7d zxid:0x3b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4331 T91 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 4350 T90 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 4369 T93 oasc.ZkController.register We are http://127.0.0.1:15097/_pq/collection1/ and leader is http://127.0.0.1:15097/_pq/collection1/
[junit4:junit4]   2> 4372 T93 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:15097/_pq
[junit4:junit4]   2> 4372 T93 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 4372 T93 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 4372 T93 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 4375 T93 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 4376 T68 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 4377 T68 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 4377 T68 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 4387 T68 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 4398 T68 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:16522/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@39e8b722
[junit4:junit4]   2> 4399 T68 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4400 T95 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:16522. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4400 T95 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:16522, initiating session
[junit4:junit4]   2> 4401 T70 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:22895
[junit4:junit4]   2> 4401 T70 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:22895
[junit4:junit4]   2> 4403 T72 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9d02617d0004 with negotiated timeout 10000 for client /140.211.11.196:22895
[junit4:junit4]   2> 4403 T95 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:16522, sessionid = 0x13c9d02617d0004, negotiated timeout = 10000
[junit4:junit4]   2> 4404 T96 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@39e8b722 name:ZooKeeperConnection Watcher:127.0.0.1:16522/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4404 T68 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4405 T68 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 4409 T68 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 4525 T68 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 4527 T68 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:34013
[junit4:junit4]   2> 4528 T68 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 4528 T68 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 4529 T68 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1359843848634
[junit4:junit4]   2> 4529 T68 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1359843848634/solr.xml
[junit4:junit4]   2> 4530 T68 oasc.CoreContainer.<init> New CoreContainer 71691097
[junit4:junit4]   2> 4530 T68 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1359843848634/'
[junit4:junit4]   2> 4531 T68 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1359843848634/'
[junit4:junit4]   2> 4568 T68 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 4569 T68 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 4569 T68 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 4569 T68 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 4570 T68 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 4570 T68 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 4571 T68 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 4571 T68 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 4571 T68 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 4572 T68 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 4590 T68 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 4608 T68 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:16522/solr
[junit4:junit4]   2> 4609 T68 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 4609 T68 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:16522 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@350aa5bc
[junit4:junit4]   2> 4610 T68 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4611 T106 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:16522. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4612 T106 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:16522, initiating session
[junit4:junit4]   2> 4612 T70 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:15084
[junit4:junit4]   2> 4613 T70 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:15084
[junit4:junit4]   2> 4625 T72 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9d02617d0005 with negotiated timeout 20000 for client /140.211.11.196:15084
[junit4:junit4]   2> 4625 T106 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:16522, sessionid = 0x13c9d02617d0005, negotiated timeout = 20000
[junit4:junit4]   2> 4626 T107 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@350aa5bc name:ZooKeeperConnection Watcher:127.0.0.1:16522 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4626 T68 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4628 T73 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c9d02617d0005
[junit4:junit4]   2> 4631 T107 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 4631 T70 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:15084 which had sessionid 0x13c9d02617d0005
[junit4:junit4]   2> 4631 T68 oaz.ZooKeeper.close Session: 0x13c9d02617d0005 closed
[junit4:junit4]   2> 4632 T68 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 4642 T68 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:16522/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@4c53953f
[junit4:junit4]   2> 4643 T68 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4643 T108 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:16522. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4644 T108 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:16522, initiating session
[junit4:junit4]   2> 4644 T70 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:57541
[junit4:junit4]   2> 4645 T70 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:57541
[junit4:junit4]   2> 4651 T72 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9d02617d0006 with negotiated timeout 20000 for client /140.211.11.196:57541
[junit4:junit4]   2> 4651 T108 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:16522, sessionid = 0x13c9d02617d0006, negotiated timeout = 20000
[junit4:junit4]   2> 4652 T109 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4c53953f name:ZooKeeperConnection Watcher:127.0.0.1:16522/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4652 T68 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4653 T73 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9d02617d0006 type:create cxid:0x1 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4660 T73 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9d02617d0006 type:create cxid:0x2 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4668 T68 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34013__pq
[junit4:junit4]   2> 4669 T73 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9d02617d0006 type:delete cxid:0x4 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:34013__pq Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:34013__pq
[junit4:junit4]   2> 4675 T68 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:34013__pq
[junit4:junit4]   2> 4682 T90 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 4684 T96 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 4686 T90 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 4691 T68 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 4698 T110 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1359843848634/collection1
[junit4:junit4]   2> 4698 T110 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 4699 T110 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 4700 T110 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 4702 T110 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1359843848634/collection1/'
[junit4:junit4]   2> 4703 T110 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty1-1359843848634/collection1/lib/README' to classloader
[junit4:junit4]   2> 4704 T110 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty1-1359843848634/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 4769 T110 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 4850 T110 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 4952 T110 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 4975 T110 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 5703 T110 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 5720 T110 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 5725 T110 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 5763 T110 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5770 T110 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5778 T110 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5782 T110 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5786 T110 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1359843848634/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty1/
[junit4:junit4]   2> 5787 T110 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@71ac4afb
[junit4:junit4]   2> 5787 T110 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 5789 T110 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty1 forceNew:false
[junit4:junit4]   2> 5789 T110 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty1
[junit4:junit4]   2> 5790 T110 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty1/index/
[junit4:junit4]   2> 5791 T110 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 5792 T110 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty1/index forceNew:false
[junit4:junit4]   2> 5805 T110 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@50eaa935; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 5806 T110 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 5807 T110 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty1/index
[junit4:junit4]   2> 5808 T110 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 5811 T110 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 5812 T110 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 5813 T110 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 5813 T110 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 5814 T110 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 5815 T110 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 5815 T110 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 5817 T110 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 5818 T110 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 5819 T110 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 5820 T110 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 5820 T110 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 5822 T110 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 5822 T110 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 5823 T110 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 5824 T110 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 5825 T110 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5826 T110 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5827 T110 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5828 T110 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5829 T110 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5830 T110 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5831 T110 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5832 T110 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5832 T110 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 5833 T110 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 5848 T110 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 5853 T110 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 5855 T110 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 5859 T110 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty1
[junit4:junit4]   2> 5860 T91 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 5861 T110 oass.SolrIndexSearcher.<init> Opening Searcher@5005d047 main
[junit4:junit4]   2> 5862 T91 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:15097__pq",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:15097/_pq"}
[junit4:junit4]   2> 5862 T110 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 5863 T110 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 5864 T110 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 5882 T90 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5882 T96 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5882 T109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5883 T110 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 5947 T111 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5005d047 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 5956 T110 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 5957 T110 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 7389 T91 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 7390 T91 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:34013__pq",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:34013/_pq"}
[junit4:junit4]   2> 7391 T91 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 1
[junit4:junit4]   2> 7391 T91 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 7397 T90 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7398 T109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7398 T96 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7961 T110 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 7961 T110 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:34013/_pq collection:collection1 shard:shard1
[junit4:junit4]   2> 7963 T110 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 7981 T73 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9d02617d0006 type:delete cxid:0x45 zxid:0x5b txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 7983 T110 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 7984 T73 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9d02617d0006 type:create cxid:0x46 zxid:0x5c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7993 T110 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 7993 T110 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 7994 T110 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:34013/_pq/collection1/
[junit4:junit4]   2> 7994 T110 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 7994 T110 oasc.SyncStrategy.syncToMe http://127.0.0.1:34013/_pq/collection1/ has no replicas
[junit4:junit4]   2> 7995 T110 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:34013/_pq/collection1/
[junit4:junit4]   2> 7995 T110 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 8002 T73 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9d02617d0006 type:create cxid:0x50 zxid:0x60 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8905 T91 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8927 T96 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8927 T90 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8927 T109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8975 T110 oasc.ZkController.register We are http://127.0.0.1:34013/_pq/collection1/ and leader is http://127.0.0.1:34013/_pq/collection1/
[junit4:junit4]   2> 8975 T110 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:34013/_pq
[junit4:junit4]   2> 8975 T110 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 8976 T110 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 8976 T110 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 8979 T110 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8981 T68 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 8982 T68 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 8982 T68 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 9187 T68 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 9191 T68 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:30729
[junit4:junit4]   2> 9192 T68 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 9192 T68 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 9193 T68 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1359843853223
[junit4:junit4]   2> 9194 T68 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1359843853223/solr.xml
[junit4:junit4]   2> 9194 T68 oasc.CoreContainer.<init> New CoreContainer 811270547
[junit4:junit4]   2> 9195 T68 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1359843853223/'
[junit4:junit4]   2> 9196 T68 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1359843853223/'
[junit4:junit4]   2> 9259 T68 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 9260 T68 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 9261 T68 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 9261 T68 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 9262 T68 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 9262 T68 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 9263 T68 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 9264 T68 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 9264 T68 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 9265 T68 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 9289 T68 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 9313 T68 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:16522/solr
[junit4:junit4]   2> 9314 T68 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 9314 T68 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:16522 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@5e84b982
[junit4:junit4]   2> 9316 T68 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 9316 T121 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:16522. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 9317 T121 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:16522, initiating session
[junit4:junit4]   2> 9317 T70 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:39639
[junit4:junit4]   2> 9318 T70 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:39639
[junit4:junit4]   2> 9320 T72 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9d02617d0007 with negotiated timeout 20000 for client /140.211.11.196:39639
[junit4:junit4]   2> 9320 T121 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:16522, sessionid = 0x13c9d02617d0007, negotiated timeout = 20000
[junit4:junit4]   2> 9321 T122 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5e84b982 name:ZooKeeperConnection Watcher:127.0.0.1:16522 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 9322 T68 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 9324 T73 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c9d02617d0007
[junit4:junit4]   2> 9326 T70 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:39639 which had sessionid 0x13c9d02617d0007
[junit4:junit4]   2> 9326 T68 oaz.ZooKeeper.close Session: 0x13c9d02617d0007 closed
[junit4:junit4]   2> 9326 T122 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 9327 T68 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 9341 T68 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:16522/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@49fa82ea
[junit4:junit4]   2> 9342 T68 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 9343 T123 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:16522. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 9343 T123 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:16522, initiating session
[junit4:junit4]   2> 9344 T70 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:19957
[junit4:junit4]   2> 9344 T70 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:19957
[junit4:junit4]   2> 9346 T72 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9d02617d0008 with negotiated timeout 20000 for client /140.211.11.196:19957
[junit4:junit4]   2> 9346 T123 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:16522, sessionid = 0x13c9d02617d0008, negotiated timeout = 20000
[junit4:junit4]   2> 9347 T124 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@49fa82ea name:ZooKeeperConnection Watcher:127.0.0.1:16522/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 9348 T68 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 9349 T73 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9d02617d0008 type:create cxid:0x1 zxid:0x6d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9351 T73 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9d02617d0008 type:create cxid:0x2 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9353 T68 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:30729__pq
[junit4:junit4]   2> 9354 T73 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9d02617d0008 type:delete cxid:0x4 zxid:0x6f txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:30729__pq Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:30729__pq
[junit4:junit4]   2> 9361 T68 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:30729__pq
[junit4:junit4]   2> 9365 T90 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 9366 T109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 9366 T96 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 9367 T96 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9370 T90 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 9370 T109 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 9372 T68 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 9382 T125 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1359843853223/collection1
[junit4:junit4]   2> 9383 T125 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 9384 T125 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 9384 T125 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 9386 T125 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1359843853223/collection1/'
[junit4:junit4]   2> 9388 T125 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty2-1359843853223/collection1/lib/README' to classloader
[junit4:junit4]   2> 9389 T125 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty2-1359843853223/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 9453 T125 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 9554 T125 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 9655 T125 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 9678 T125 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 10430 T125 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 10441 T125 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 10443 T91 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 10444 T91 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:34013__pq",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:34013/_pq"}
[junit4:junit4]   2> 10445 T125 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 10479 T96 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10479 T109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10479 T124 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10479 T90 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10501 T125 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 10506 T125 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 10511 T125 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 10513 T125 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 10516 T125 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1359843853223/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty2/
[junit4:junit4]   2> 10516 T125 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@71ac4afb
[junit4:junit4]   2> 10517 T125 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 10518 T125 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty2 forceNew:false
[junit4:junit4]   2> 10518 T125 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty2
[junit4:junit4]   2> 10518 T125 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty2/index/
[junit4:junit4]   2> 10519 T125 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 10520 T125 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty2/index forceNew:false
[junit4:junit4]   2> 10526 T125 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4984e7c9; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 10526 T125 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 10527 T125 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty2/index
[junit4:junit4]   2> 10527 T125 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 10529 T125 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 10530 T125 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 10530 T125 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 10531 T125 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 10531 T125 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 10532 T125 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 10532 T125 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 10533 T125 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 10534 T125 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 10534 T125 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 10535 T125 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 10535 T125 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 10536 T125 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 10536 T125 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 10537 T125 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 10537 T125 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 10538 T125 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10538 T125 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10539 T125 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10539 T125 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10540 T125 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10541 T125 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10541 T125 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10542 T125 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 10542 T125 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 10543 T125 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 10553 T125 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 10557 T125 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 10559 T125 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 10562 T125 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty2
[junit4:junit4]   2> 10563 T125 oass.SolrIndexSearcher.<init> Opening Searcher@1d531b10 main
[junit4:junit4]   2> 10564 T125 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 10565 T125 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 10565 T125 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 10582 T125 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 10643 T126 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1d531b10 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 10648 T125 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 10648 T125 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 11989 T91 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11991 T91 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:30729__pq",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:30729/_pq"}
[junit4:junit4]   2> 11991 T91 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4]   2> 11991 T91 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 12003 T90 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12003 T109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12003 T96 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12003 T124 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12653 T125 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 12653 T125 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:30729/_pq collection:collection1 shard:shard1
[junit4:junit4]   2> 12659 T125 oasc.ZkController.register We are http://127.0.0.1:30729/_pq/collection1/ and leader is http://127.0.0.1:34013/_pq/collection1/
[junit4:junit4]   2> 12659 T125 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:30729/_pq
[junit4:junit4]   2> 12660 T125 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 12660 T125 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C4 name=collection1 org.apache.solr.core.SolrCore@110bc24b url=http://127.0.0.1:30729/_pq/collection1 node=127.0.0.1:30729__pq C4_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:30729__pq, base_url=http://127.0.0.1:30729/_pq}
[junit4:junit4]   2> 12666 T127 C4 P30729 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 12667 T125 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12668 T127 C4 P30729 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 12668 T127 C4 P30729 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 12669 T127 C4 P30729 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 12669 T68 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 12670 T68 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 12670 T68 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 12671 T127 C4 P30729 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 12858 T68 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 12861 T68 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:32616
[junit4:junit4]   2> 12862 T68 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 12862 T68 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 12863 T68 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1359843856916
[junit4:junit4]   2> 12863 T68 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1359843856916/solr.xml
[junit4:junit4]   2> 12864 T68 oasc.CoreContainer.<init> New CoreContainer 1935102971
[junit4:junit4]   2> 12865 T68 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1359843856916/'
[junit4:junit4]   2> 12865 T68 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1359843856916/'
[junit4:junit4]   2> 12918 T68 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 12919 T68 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 12920 T68 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 12920 T68 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 12921 T68 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 12921 T68 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 12922 T68 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 12922 T68 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 12923 T68 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 12924 T68 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 12948 T68 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 12970 T68 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:16522/solr
[junit4:junit4]   2> 12971 T68 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 12971 T68 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:16522 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@1169d3a
[junit4:junit4]   2> 12972 T68 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 12973 T137 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:16522. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 12974 T137 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:16522, initiating session
[junit4:junit4]   2> 12974 T70 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:20328
[junit4:junit4]   2> 12975 T70 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:20328
[junit4:junit4]   2> 12977 T72 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9d02617d0009 with negotiated timeout 20000 for client /140.211.11.196:20328
[junit4:junit4]   2> 12977 T137 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:16522, sessionid = 0x13c9d02617d0009, negotiated timeout = 20000
[junit4:junit4]   2> 12978 T138 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1169d3a name:ZooKeeperConnection Watcher:127.0.0.1:16522 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 12978 T68 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 12980 T73 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c9d02617d0009
[junit4:junit4]   2> 12981 T138 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 12981 T68 oaz.ZooKeeper.close Session: 0x13c9d02617d0009 closed
[junit4:junit4]   2> 12981 T70 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:20328 which had sessionid 0x13c9d02617d0009
[junit4:junit4]   2> 12982 T68 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 12996 T68 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:16522/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@af90d11
[junit4:junit4]   2> 12997 T68 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 12998 T139 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:16522. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 12999 T139 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:16522, initiating session
[junit4:junit4]   2> 12999 T70 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:16080
[junit4:junit4]   2> 13000 T70 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:16080
[junit4:junit4]   2> 13002 T72 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9d02617d000a with negotiated timeout 20000 for client /140.211.11.196:16080
[junit4:junit4]   2> 13002 T139 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:16522, sessionid = 0x13c9d02617d000a, negotiated timeout = 20000
[junit4:junit4]   2> 13002 T140 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@af90d11 name:ZooKeeperConnection Watcher:127.0.0.1:16522/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 13003 T68 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 13004 T73 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9d02617d000a type:create cxid:0x1 zxid:0x80 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 13006 T73 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9d02617d000a type:create cxid:0x2 zxid:0x81 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 13009 T68 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:32616__pq
[junit4:junit4]   2> 13010 T73 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9d02617d000a type:delete cxid:0x4 zxid:0x82 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:32616__pq Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:32616__pq
[junit4:junit4]   2> 13016 T68 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:32616__pq
[junit4:junit4]   2> 13020 T90 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13020 T109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13021 T96 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 13021 T124 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 13022 T96 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13022 T124 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13024 T90 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 13024 T109 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 13028 T68 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 13038 T141 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1359843856916/collection1
[junit4:junit4]   2> 13038 T141 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 13040 T141 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 13040 T141 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 13042 T141 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1359843856916/collection1/'
[junit4:junit4]   2> 13044 T141 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty3-1359843856916/collection1/lib/README' to classloader
[junit4:junit4]   2> 13045 T141 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty3-1359843856916/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 13119 T141 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 13200 T141 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 13301 T141 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 13318 T141 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 13516 T91 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 13518 T91 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:30729__pq",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:30729/_pq"}
[junit4:junit4]   2> 13531 T90 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13531 T109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13531 T140 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13531 T96 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13531 T124 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13986 T141 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 13997 T141 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 14000 T141 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 14027 T141 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 14032 T141 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 14036 T141 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 14039 T141 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 14041 T141 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1359843856916/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty3/
[junit4:junit4]   2> 14041 T141 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@71ac4afb
[junit4:junit4]   2> 14042 T141 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 14043 T141 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty3 forceNew:false
[junit4:junit4]   2> 14043 T141 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty3
[junit4:junit4]   2> 14044 T141 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty3/index/
[junit4:junit4]   2> 14044 T141 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 14045 T141 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty3/index forceNew:false
[junit4:junit4]   2> 14049 T141 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1b8fc338; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 14050 T141 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 14051 T141 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty3/index
[junit4:junit4]   2> 14051 T141 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 14053 T141 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 14053 T141 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 14054 T141 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 14054 T141 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 14055 T141 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 14055 T141 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 14055 T141 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 14056 T141 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 14056 T141 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 14057 T141 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 14058 T141 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 14058 T141 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 14059 T141 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 14059 T141 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 14059 T141 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 14060 T141 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 14060 T141 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14061 T141 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14062 T141 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14062 T141 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14063 T141 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14063 T141 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14064 T141 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14064 T141 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14065 T141 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 14066 T141 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 14074 T141 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 14077 T141 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 14079 T141 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 14081 T141 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty3
[junit4:junit4]   2> 14082 T141 oass.SolrIndexSearcher.<init> Opening Searcher@1b16b24f main
[junit4:junit4]   2> 14083 T141 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 14083 T141 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 14084 T141 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 14099 T141 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 14141 T142 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1b16b24f main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 14147 T141 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 14147 T141 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 15040 T91 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 15042 T91 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:32616__pq",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:32616/_pq"}
[junit4:junit4]   2> 15042 T91 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4]   2> 15042 T91 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 15091 T90 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15091 T124 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15091 T140 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15091 T109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15091 T96 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15150 T141 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 15150 T141 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:32616/_pq collection:collection1 shard:shard1
[junit4:junit4]   2> 15155 T141 oasc.ZkController.register We are http://127.0.0.1:32616/_pq/collection1/ and leader is http://127.0.0.1:34013/_pq/collection1/
[junit4:junit4]   2> 15156 T141 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:32616/_pq
[junit4:junit4]   2> 15156 T141 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 15156 T141 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C5 name=collection1 org.apache.solr.core.SolrCore@622daa46 url=http://127.0.0.1:32616/_pq/collection1 node=127.0.0.1:32616__pq C5_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:32616__pq, base_url=http://127.0.0.1:32616/_pq}
[junit4:junit4]   2> 15157 T143 C5 P32616 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 15158 T143 C5 P32616 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 15158 T141 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 15158 T143 C5 P32616 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 15159 T143 C5 P32616 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 15160 T68 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 15161 T68 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 15161 T68 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 15191 T143 C5 P32616 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 15355 T68 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 15358 T68 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:35524
[junit4:junit4]   2> 15359 T68 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 15360 T68 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 15360 T68 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1359843859401
[junit4:junit4]   2> 15361 T68 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1359843859401/solr.xml
[junit4:junit4]   2> 15361 T68 oasc.CoreContainer.<init> New CoreContainer 901305989
[junit4:junit4]   2> 15362 T68 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1359843859401/'
[junit4:junit4]   2> 15363 T68 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1359843859401/'
[junit4:junit4]   2> 15418 T68 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 15419 T68 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 15420 T68 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 15420 T68 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 15421 T68 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 15421 T68 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 15422 T68 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 15423 T68 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 15423 T68 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 15424 T68 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 15446 T68 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 15466 T68 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:16522/solr
[junit4:junit4]   2> 15467 T68 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 15468 T68 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:16522 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@2b13e30a
[junit4:junit4]   2> 15469 T68 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 15469 T153 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:16522. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 15470 T153 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:16522, initiating session
[junit4:junit4]   2> 15471 T70 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:25257
[junit4:junit4]   2> 15471 T70 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:25257
[junit4:junit4]   2> 15474 T72 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9d02617d000b with negotiated timeout 20000 for client /140.211.11.196:25257
[junit4:junit4]   2> 15474 T153 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:16522, sessionid = 0x13c9d02617d000b, negotiated timeout = 20000
[junit4:junit4]   2> 15474 T154 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2b13e30a name:ZooKeeperConnection Watcher:127.0.0.1:16522 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 15475 T68 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 15477 T73 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c9d02617d000b
[junit4:junit4]   2> 15497 T154 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 15497 T68 oaz.ZooKeeper.close Session: 0x13c9d02617d000b closed
[junit4:junit4]   2> 15497 T70 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:25257 which had sessionid 0x13c9d02617d000b
[junit4:junit4]   2> 15498 T68 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 15510 T68 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:16522/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@7c7d3eb3
[junit4:junit4]   2> 15511 T68 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 15511 T155 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:16522. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 15512 T155 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:16522, initiating session
[junit4:junit4]   2> 15512 T70 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:31821
[junit4:junit4]   2> 15513 T70 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:31821
[junit4:junit4]   2> 15515 T72 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9d02617d000c with negotiated timeout 20000 for client /140.211.11.196:31821
[junit4:junit4]   2> 15515 T155 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:16522, sessionid = 0x13c9d02617d000c, negotiated timeout = 20000
[junit4:junit4]   2> 15515 T156 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7c7d3eb3 name:ZooKeeperConnection Watcher:127.0.0.1:16522/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 15516 T68 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 15517 T73 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9d02617d000c type:create cxid:0x1 zxid:0x93 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 15519 T73 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9d02617d000c type:create cxid:0x2 zxid:0x94 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 15521 T68 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35524__pq
[junit4:junit4]   2> 15522 T73 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9d02617d000c type:delete cxid:0x4 zxid:0x95 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:35524__pq Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:35524__pq
[junit4:junit4]   2> 15530 T68 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:35524__pq
[junit4:junit4]   2> 15533 T90 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15533 T140 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15533 T109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15534 T124 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 15534 T96 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 15535 T124 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15535 T96 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15537 T90 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 15537 T140 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 15539 T109 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 15541 T68 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 15551 T157 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1359843859401/collection1
[junit4:junit4]   2> 15551 T157 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 15552 T157 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 15552 T157 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 15555 T157 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1359843859401/collection1/'
[junit4:junit4]   2> 15556 T157 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty4-1359843859401/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 15557 T157 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty4-1359843859401/collection1/lib/README' to classloader
[junit4:junit4]   2> 15616 T157 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 15694 T157 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2>  C4_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:30729__pq, base_url=http://127.0.0.1:30729/_pq}
[junit4:junit4]   2> 15715 T127 C4 P30729 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:34013/_pq/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 15725 T127 C4 P30729 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 15743 T127 C4 P30729 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:30729/_pq START replicas=[http://127.0.0.1:34013/_pq/collection1/] nUpdates=100
[junit4:junit4]   2> 15754 T127 C4 P30729 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 15755 T127 C4 P30729 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 15755 T127 C4 P30729 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 15755 T127 C4 P30729 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 15756 T127 C4 P30729 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 15756 T127 C4 P30729 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:34013/_pq/collection1/. core=collection1
[junit4:junit4]   2> 15757 T127 C4 P30729 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C6 name=collection1 org.apache.solr.core.SolrCore@4dc7bae0 url=http://127.0.0.1:34013/_pq/collection1 node=127.0.0.1:34013__pq C6_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:34013__pq, base_url=http://127.0.0.1:34013/_pq, leader=true}
[junit4:junit4]   2> 15765 T101 C6 P34013 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15787 T102 C6 P34013 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 15788 T102 C6 P34013 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty1
[junit4:junit4]   2> 15793 T102 C6 P34013 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@50eaa935; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 15794 T102 C6 P34013 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 15795 T157 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 15800 T102 C6 P34013 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@50eaa935; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@50eaa935; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 15800 T102 C6 P34013 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 15802 T102 C6 P34013 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty1
[junit4:junit4]   2> 15803 T102 C6 P34013 oass.SolrIndexSearcher.<init> Opening Searcher@36160b84 realtime
[junit4:junit4]   2> 15803 T102 C6 P34013 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15804 T102 C6 P34013 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 34
[junit4:junit4]   2> 15805 T127 C4 P30729 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 15806 T127 C4 P30729 oascsi.HttpClientUtil.createClient Creating new http client, config:connTimeout=5000&socketTimeout=20000&allowCompression=false&maxConnections=10000&maxConnectionsPerHost=10000
[junit4:junit4]   2> 15817 T157 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 15834 T127 C4 P30729 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 15838 T103 C6 P34013 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 15839 T103 C6 P34013 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=3 
[junit4:junit4]   2> 15841 T127 C4 P30729 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 15841 T127 C4 P30729 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 15841 T127 C4 P30729 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 15845 T103 C6 P34013 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty1
[junit4:junit4]   2> 15846 T103 C6 P34013 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty1/index
[junit4:junit4]   2> 15846 T103 C6 P34013 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=2 
[junit4:junit4]   2> 15848 T127 C4 P30729 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 15850 T127 C4 P30729 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty2/index.20130202162420072 forceNew:false
[junit4:junit4]   2> 15851 T127 C4 P30729 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty2
[junit4:junit4]   2> 15852 T127 C4 P30729 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty2/index.20130202162420072 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3566416b; maxCacheMB=48.0 maxMergeSizeMB=4.0)) fullCopy=true
[junit4:junit4]   2> 15868 T103 C6 P34013 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=5 
[junit4:junit4]   2> 15884 T127 C4 P30729 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty2/index
[junit4:junit4]   2> 15885 T127 C4 P30729 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 15885 T127 C4 P30729 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130202162420072
[junit4:junit4]   2> 15892 T127 C4 P30729 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty2
[junit4:junit4]   2> 15904 T127 C4 P30729 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty2
[junit4:junit4]   2> 15905 T127 C4 P30729 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty2/index/ new=./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty2/index.20130202162420072
[junit4:junit4]   2> 15910 T127 C4 P30729 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty2/index.20130202162420072 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3566416b; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 15911 T127 C4 P30729 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 15912 T127 C4 P30729 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty2
[junit4:junit4]   2> 15918 T127 C4 P30729 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 15918 T127 C4 P30729 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 15918 T127 C4 P30729 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 15920 T127 C4 P30729 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty2/index.20130202162420072
[junit4:junit4]   2> 15921 T127 C4 P30729 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty2
[junit4:junit4]   2> 15925 T127 C4 P30729 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty2/index.20130202162420072 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3566416b; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 15926 T127 C4 P30729 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 15926 T127 C4 P30729 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 15927 T127 C4 P30729 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty2
[junit4:junit4]   2> 15928 T127 C4 P30729 oass.SolrIndexSearcher.<init> Opening Searcher@a5bc74f main
[junit4:junit4]   2> 15930 T126 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@a5bc74f main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 15931 T126 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty2/index
[junit4:junit4]   2> 15931 T127 C4 P30729 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 15936 T127 C4 P30729 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty2/index.20130202162420072 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3566416b; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty2/index.20130202162420072 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3566416b; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 15936 T127 C4 P30729 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 15938 T127 C4 P30729 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty2
[junit4:junit4]   2> 15938 T127 C4 P30729 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15938 T127 C4 P30729 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty2/index.20130202162420072
[junit4:junit4]   2> 15939 T127 C4 P30729 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty2/index
[junit4:junit4]   2> 15939 T127 C4 P30729 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 15939 T127 C4 P30729 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 15940 T127 C4 P30729 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 15940 T127 C4 P30729 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 15942 T127 C4 P30729 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 16481 T157 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 16491 T157 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 16494 T157 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 16520 T157 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 16532 T157 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 16536 T157 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 16538 T157 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 16541 T157 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1359843859401/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty4/
[junit4:junit4]   2> 16541 T157 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@71ac4afb
[junit4:junit4]   2> 16542 T157 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 16542 T157 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty4 forceNew:false
[junit4:junit4]   2> 16543 T157 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty4
[junit4:junit4]   2> 16543 T157 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty4/index/
[junit4:junit4]   2> 16543 T157 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 16544 T157 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty4/index forceNew:false
[junit4:junit4]   2> 16552 T157 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4b4f536b; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 16553 T157 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 16554 T157 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty4/index
[junit4:junit4]   2> 16554 T157 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 16556 T157 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 16556 T157 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 16557 T157 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 16557 T157 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 16558 T157 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 16558 T157 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 16558 T157 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 16559 T157 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 16559 T157 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 16560 T157 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 16561 T157 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 16561 T157 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 16562 T157 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 16562 T157 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 16562 T157 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 16563 T157 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 16563 T157 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16564 T157 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16565 T157 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16565 T157 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16566 T157 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16566 T157 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16567 T157 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16567 T157 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16568 T157 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 16569 T157 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 16577 T157 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 16580 T157 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 16582 T157 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 16584 T157 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty4
[junit4:junit4]   2> 16585 T157 oass.SolrIndexSearcher.<init> Opening Searcher@4474ce1f main
[junit4:junit4]   2> 16586 T157 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 16587 T157 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 16587 T157 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 16593 T157 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 16615 T91 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16616 T91 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:32616__pq",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:32616/_pq"}
[junit4:junit4]   2> 16629 T91 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:30729__pq",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:30729/_pq"}
[junit4:junit4]   2> 16644 T160 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4474ce1f main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 16646 T156 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16646 T124 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16646 T96 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16646 T109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16646 T140 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16646 T90 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16651 T157 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 16652 T157 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 18162 T91 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 18163 T91 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:35524__pq",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:35524/_pq"}
[junit4:junit4]   2> 18164 T91 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4]   2> 18164 T91 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 18201 T90 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18201 T156 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18201 T109 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18201 T140 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18201 T96 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18201 T124 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18660 T157 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 18660 T157 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:35524/_pq collection:collection1 shard:shard1
[junit4:junit4]   2> 18666 T157 oasc.ZkController.register We are http://127.0.0.1:35524/_pq/collection1/ and leader is http://127.0.0.1:34013/_pq/collection1/
[junit4:junit4]   2> 18666 T157 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:35524/_pq
[junit4:junit4]   2> 18667 T157 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 18667 T157 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C7 name=collection1 org.apache.solr.core.SolrCore@484bedfb url=http://127.0.0.1:35524/_pq/collection1 node=127.0.0.1:35524__pq C7_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:35524__pq, base_url=http://127.0.0.1:35524/_pq}
[junit4:junit4]   2> 18668 T161 C7 P35524 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 18669 T161 C7 P35524 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 18669 T157 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 18669 T161 C7 P35524 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 18669 T161 C7 P35524 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 18670 T68 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 18671 T68 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 18672 T161 C7 P35524 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 18672 T68 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 18691 T68 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 18694 T68 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4]   2> 18694 T68 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4]   2> 18696 T68 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C5_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:32616__pq, base_url=http://127.0.0.1:32616/_pq}
[junit4:junit4]   2> 19218 T143 C5 P32616 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:34013/_pq/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 19218 T143 C5 P32616 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:32616/_pq START replicas=[http://127.0.0.1:34013/_pq/collection1/] nUpdates=100
[junit4:junit4]   2> 19219 T143 C5 P32616 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 19219 T143 C5 P32616 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 19220 T143 C5 P32616 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 19220 T143 C5 P32616 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 19220 T143 C5 P32616 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 19221 T143 C5 P32616 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:34013/_pq/collection1/. core=collection1
[junit4:junit4]   2> 19221 T143 C5 P32616 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 19223 T101 C6 P34013 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 19235 T105 C6 P34013 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 19243 T105 C6 P34013 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@50eaa935; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@50eaa935; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 19244 T105 C6 P34013 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 19245 T105 C6 P34013 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty1
[junit4:junit4]   2> 19245 T105 C6 P34013 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 19246 T105 C6 P34013 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 11
[junit4:junit4]   2> 19247 T143 C5 P32616 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 19247 T143 C5 P32616 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 19251 T103 C6 P34013 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 19252 T143 C5 P32616 oash.SnapPuller.fetchLatestIndex Master's generation: 3
[junit4:junit4]   2> 19252 T143 C5 P32616 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 19253 T143 C5 P32616 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 19256 T103 C6 P34013 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty1
[junit4:junit4]   2> 19257 T103 C6 P34013 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty1/index
[junit4:junit4]   2> 19257 T103 C6 P34013 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=3&version=2} status=0 QTime=2 
[junit4:junit4]   2> 19258 T143 C5 P32616 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 19260 T143 C5 P32616 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty3/index.20130202162423483 forceNew:false
[junit4:junit4]   2> 19260 T143 C5 P32616 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359843844230/jetty3
[junit4:junit4]   2> 19261 T143 C5 P32616 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lu

[...truncated too long message...]

)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4:junit4]   2> 	        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]   2> 	        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  25) Thread[id=980, name=HashSessionScavenger-26, state=TIMED_WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at java.lang.Object.wait(Native Method)
[junit4:junit4]   2> 	        at java.util.TimerThread.mainLoop(Timer.java:552)
[junit4:junit4]   2> 	        at java.util.TimerThread.run(Timer.java:505)
[junit4:junit4]   2> 	  26) Thread[id=991, name=TEST-BasicDistributedZkTest.testDistribSearch-seed#[68A457610324D72F]-SendThread(localhost:59545), state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.nio.ch.KQueueArrayWrapper.interrupt(Native Method)
[junit4:junit4]   2> 	        at sun.nio.ch.KQueueArrayWrapper.interrupt(KQueueArrayWrapper.java:164)
[junit4:junit4]   2> 	        at sun.nio.ch.KQueueSelectorImpl.wakeup(KQueueSelectorImpl.java:245)
[junit4:junit4]   2> 	        at java.nio.channels.spi.AbstractSelector$1.interrupt(AbstractSelector.java:210)
[junit4:junit4]   2> 	        at java.nio.channels.spi.AbstractSelector.begin(AbstractSelector.java:216)
[junit4:junit4]   2> 	        at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:102)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
[junit4:junit4]   2> 	        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)
[junit4:junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
[junit4:junit4]   2> 	  27) Thread[id=994, name=searcherExecutor-775-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> NOTE: test params are: codec=DummyCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=DUMMY, chunkSize=63), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=DUMMY, chunkSize=63)), sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=el_CY, timezone=Etc/GMT-9
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=28,free=48050376,total=284164096
[junit4:junit4]   2> NOTE: All tests run in this JVM: [FastVectorHighlighterTest, DistanceFunctionTest, TestQuerySenderNoQuery, CircularListTest, ExternalFileFieldSortTest, ZkControllerTest, MBeansHandlerTest, DateMathParserTest, UpdateRequestProcessorFactoryTest, SampleTest, TestAtomicUpdateErrorCases, FieldAnalysisRequestHandlerTest, TestLFUCache, SolrCoreCheckLockOnStartupTest, LeaderElectionIntegrationTest, SuggesterFSTTest, TestFastOutputStream, MinimalSchemaTest, NumericFieldsTest, SpellPossibilityIteratorTest, FileBasedSpellCheckerTest, CopyFieldTest, TestBinaryResponseWriter, TestSearchPerf, TestNumberUtils, SolrRequestParserTest, RequestHandlersTest, SortByFunctionTest, SuggesterWFSTTest, AutoCommitTest, TestLMDirichletSimilarityFactory, TestLazyCores, ChaosMonkeySafeLeaderTest, SolrCmdDistributorTest, TermsComponentTest, TestFuzzyAnalyzedSuggestions, TestDocumentBuilder, TestCoreContainer, PreAnalyzedFieldTest, TestQueryTypes, PluginInfoTest, DateFieldTest, ChaosMonkeyNothingIsSafeTest, TestZkChroot, DirectSolrConnectionTest, DistributedSpellCheckComponentTest, TestJmxMonitoredMap, AnalysisAfterCoreReloadTest, DefaultValueUpdateProcessorTest, DistributedTermsComponentTest, TestReload, TestCharFilters, BinaryUpdateRequestHandlerTest, TestLMJelinekMercerSimilarityFactory, SpatialFilterTest, TestLuceneMatchVersion, TestPartialUpdateDeduplication, TestSuggestSpellingConverter, NotRequiredUniqueKeyTest, TestFaceting, BasicDistributedZkTest]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=BasicDistributedZkTest -Dtests.seed=68A457610324D72F -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=el_CY -Dtests.timezone=Etc/GMT-9 -Dtests.file.encoding=US-ASCII
[junit4:junit4] ERROR   0.00s J0 | BasicDistributedZkTest (suite) <<<
[junit4:junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([68A457610324D72F]:0)
[junit4:junit4] Completed on J0 in 7223.62s, 1 test, 2 errors <<< FAILURES!

[...truncated 12 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:346: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/common-build.xml:428: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:1176: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:841: There were test failures: 249 suites, 1038 tests, 1 suite-level error, 1 error, 1 failure, 23 ignored (5 assumptions)

Total time: 149 minutes 16 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-trunk-java7 - Build # 3702 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-java7/3702/

1 tests failed.
REGRESSION:  org.apache.solr.cloud.SyncSliceTest.testDistribSearch

Error Message:
shard1 should have just been set up to be inconsistent - but it's still consistent

Stack Trace:
java.lang.AssertionError: shard1 should have just been set up to be inconsistent - but it's still consistent
	at __randomizedtesting.SeedInfo.seed([25484D8AD1576AAC:A4AEC392A6080A90]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNotNull(Assert.java:526)
	at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:214)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:794)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:722)




Build Log:
[...truncated 9412 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.SyncSliceTest
[junit4:junit4]   2> 0 T2697 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /h_vtp/t
[junit4:junit4]   2> 6 T2697 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./solrtest-SyncSliceTest-1359816869628
[junit4:junit4]   2> 7 T2697 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 8 T2698 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 11 T2698 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 11 T2698 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4]   2> 11 T2698 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4]   2> 12 T2698 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 13 T2698 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4]   2> 108 T2697 oasc.ZkTestServer.run start zk server on port:21162
[junit4:junit4]   2> 108 T2697 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:21162 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@3933c5d4
[junit4:junit4]   2> 109 T2697 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 109 T2703 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:21162. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 110 T2703 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:21162, initiating session
[junit4:junit4]   2> 110 T2699 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:18776
[junit4:junit4]   2> 111 T2699 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:18776
[junit4:junit4]   2> 111 T2701 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 114 T2701 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9b66c7020000 with negotiated timeout 10000 for client /140.211.11.196:18776
[junit4:junit4]   2> 114 T2703 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:21162, sessionid = 0x13c9b66c7020000, negotiated timeout = 10000
[junit4:junit4]   2> 114 T2704 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3933c5d4 name:ZooKeeperConnection Watcher:127.0.0.1:21162 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 114 T2697 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 115 T2697 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 118 T2702 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c9b66c7020000
[junit4:junit4]   2> 129 T2704 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 130 T2699 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:18776 which had sessionid 0x13c9b66c7020000
[junit4:junit4]   2> 129 T2697 oaz.ZooKeeper.close Session: 0x13c9b66c7020000 closed
[junit4:junit4]   2> 131 T2697 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:21162/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@7a13b09f
[junit4:junit4]   2> 132 T2697 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 132 T2705 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:21162. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 132 T2705 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:21162, initiating session
[junit4:junit4]   2> 133 T2699 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:31986
[junit4:junit4]   2> 133 T2699 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:31986
[junit4:junit4]   2> 134 T2701 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9b66c7020001 with negotiated timeout 10000 for client /140.211.11.196:31986
[junit4:junit4]   2> 134 T2705 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:21162, sessionid = 0x13c9b66c7020001, negotiated timeout = 10000
[junit4:junit4]   2> 135 T2706 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7a13b09f name:ZooKeeperConnection Watcher:127.0.0.1:21162/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 135 T2697 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 136 T2697 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 140 T2697 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 148 T2697 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 151 T2697 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 154 T2697 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 155 T2697 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 173 T2697 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 174 T2697 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 278 T2697 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 279 T2697 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 283 T2697 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 284 T2697 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 294 T2697 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 295 T2697 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 298 T2697 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 298 T2697 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 302 T2697 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 303 T2697 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 306 T2697 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 307 T2697 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 310 T2697 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 311 T2697 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 313 T2702 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c9b66c7020001
[junit4:junit4]   2> 320 T2706 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 320 T2699 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:31986 which had sessionid 0x13c9b66c7020001
[junit4:junit4]   2> 320 T2697 oaz.ZooKeeper.close Session: 0x13c9b66c7020001 closed
[junit4:junit4]   2> 486 T2697 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 491 T2697 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:56011
[junit4:junit4]   2> 492 T2697 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 493 T2697 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 493 T2697 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1359816869942
[junit4:junit4]   2> 494 T2697 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1359816869942/solr.xml
[junit4:junit4]   2> 494 T2697 oasc.CoreContainer.<init> New CoreContainer 289000432
[junit4:junit4]   2> 495 T2697 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1359816869942/'
[junit4:junit4]   2> 496 T2697 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1359816869942/'
[junit4:junit4]   2> 563 T2697 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 564 T2697 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 564 T2697 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 565 T2697 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 565 T2697 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 566 T2697 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 566 T2697 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 567 T2697 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 567 T2697 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 568 T2697 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 592 T2697 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 614 T2697 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:21162/solr
[junit4:junit4]   2> 615 T2697 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 615 T2697 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:21162 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@12006d7b
[junit4:junit4]   2> 616 T2697 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 617 T2716 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:21162. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 617 T2716 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:21162, initiating session
[junit4:junit4]   2> 618 T2699 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:32170
[junit4:junit4]   2> 618 T2699 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:32170
[junit4:junit4]   2> 622 T2701 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9b66c7020002 with negotiated timeout 20000 for client /140.211.11.196:32170
[junit4:junit4]   2> 623 T2716 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:21162, sessionid = 0x13c9b66c7020002, negotiated timeout = 20000
[junit4:junit4]   2> 623 T2717 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@12006d7b name:ZooKeeperConnection Watcher:127.0.0.1:21162 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 623 T2697 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 625 T2702 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c9b66c7020002
[junit4:junit4]   2> 633 T2699 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:32170 which had sessionid 0x13c9b66c7020002
[junit4:junit4]   2> 633 T2717 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 633 T2697 oaz.ZooKeeper.close Session: 0x13c9b66c7020002 closed
[junit4:junit4]   2> 635 T2697 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 647 T2697 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:21162/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@2292f6d9
[junit4:junit4]   2> 648 T2697 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 649 T2718 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:21162. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 649 T2718 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:21162, initiating session
[junit4:junit4]   2> 650 T2699 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:20410
[junit4:junit4]   2> 650 T2699 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:20410
[junit4:junit4]   2> 658 T2701 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9b66c7020003 with negotiated timeout 20000 for client /140.211.11.196:20410
[junit4:junit4]   2> 658 T2718 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:21162, sessionid = 0x13c9b66c7020003, negotiated timeout = 20000
[junit4:junit4]   2> 659 T2719 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2292f6d9 name:ZooKeeperConnection Watcher:127.0.0.1:21162/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 659 T2697 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 668 T2702 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9b66c7020003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 689 T2697 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 692 T2697 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56011_h_vtp%2Ft
[junit4:junit4]   2> 693 T2702 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9b66c7020003 type:delete cxid:0x7 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:56011_h_vtp%2Ft Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:56011_h_vtp%2Ft
[junit4:junit4]   2> 694 T2697 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:56011_h_vtp%2Ft
[junit4:junit4]   2> 704 T2697 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 730 T2702 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9b66c7020003 type:delete cxid:0x15 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 737 T2697 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 746 T2697 oasc.Overseer.start Overseer (id=89116958368333827-127.0.0.1:56011_h_vtp%2Ft-n_0000000000) starting
[junit4:junit4]   2> 747 T2702 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9b66c7020003 type:create cxid:0x1a zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 756 T2702 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9b66c7020003 type:create cxid:0x1b zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 758 T2702 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9b66c7020003 type:create cxid:0x1c zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 764 T2702 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9b66c7020003 type:create cxid:0x1d zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 766 T2721 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 767 T2697 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 777 T2697 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 785 T2722 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1359816869942/collection1
[junit4:junit4]   2> 785 T2720 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 786 T2722 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 786 T2722 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 787 T2722 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 788 T2722 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1359816869942/collection1/'
[junit4:junit4]   2> 789 T2722 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-controljetty-1359816869942/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 790 T2722 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-controljetty-1359816869942/collection1/lib/README' to classloader
[junit4:junit4]   2> 845 T2722 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 919 T2722 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1019 T2722 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 1026 T2722 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 1681 T2722 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1692 T2722 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1696 T2722 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1710 T2722 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1715 T2722 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1720 T2722 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1721 T2722 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1722 T2722 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-controljetty-1359816869942/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1359816869627/control/data/
[junit4:junit4]   2> 1722 T2722 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@12bef803
[junit4:junit4]   2> 1723 T2722 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 1724 T2722 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/control/data forceNew:false
[junit4:junit4]   2> 1725 T2722 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/control/data
[junit4:junit4]   2> 1725 T2722 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1359816869627/control/data/index/
[junit4:junit4]   2> 1726 T2722 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1359816869627/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1727 T2722 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/control/data/index forceNew:false
[junit4:junit4]   2> 1734 T2722 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359816869627/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@25b6aed5; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 1734 T2722 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 1735 T2722 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/control/data/index
[junit4:junit4]   2> 1736 T2722 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 1738 T2722 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1738 T2722 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 1738 T2722 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1739 T2722 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 1739 T2722 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1739 T2722 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1740 T2722 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1740 T2722 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1741 T2722 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1741 T2722 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 1742 T2722 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 1742 T2722 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1743 T2722 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 1743 T2722 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 1743 T2722 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 1744 T2722 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1744 T2722 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1745 T2722 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1745 T2722 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1746 T2722 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1746 T2722 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1747 T2722 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1747 T2722 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1747 T2722 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1748 T2722 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 1748 T2722 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 1762 T2722 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1766 T2722 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 1768 T2722 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 1770 T2722 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/control/data
[junit4:junit4]   2> 1772 T2722 oass.SolrIndexSearcher.<init> Opening Searcher@5d54c58 main
[junit4:junit4]   2> 1773 T2722 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1773 T2722 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1774 T2722 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 1782 T2722 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 1841 T2723 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5d54c58 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1845 T2722 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 1846 T2722 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 1846 T2702 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9b66c7020003 type:create cxid:0x50 zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 2289 T2720 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2290 T2720 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:56011_h_vtp%2Ft",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:56011/h_vtp/t"}
[junit4:junit4]   2> 2291 T2720 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4]   2> 2291 T2720 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 2291 T2702 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9b66c7020003 type:create cxid:0x58 zxid:0x2c txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 2307 T2719 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 2855 T2722 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 2855 T2722 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:56011/h_vtp/t collection:control_collection shard:shard1
[junit4:junit4]   2> 2856 T2722 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4]   2> 2876 T2702 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9b66c7020003 type:delete cxid:0x72 zxid:0x36 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4]   2> 2877 T2722 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 2878 T2702 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9b66c7020003 type:create cxid:0x73 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2880 T2722 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 2880 T2722 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 2880 T2722 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:56011/h_vtp/t/collection1/
[junit4:junit4]   2> 2881 T2722 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 2881 T2722 oasc.SyncStrategy.syncToMe http://127.0.0.1:56011/h_vtp/t/collection1/ has no replicas
[junit4:junit4]   2> 2881 T2722 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:56011/h_vtp/t/collection1/
[junit4:junit4]   2> 2881 T2722 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4]   2> 2885 T2702 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9b66c7020003 type:create cxid:0x7d zxid:0x3b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3811 T2720 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3871 T2719 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 3907 T2722 oasc.ZkController.register We are http://127.0.0.1:56011/h_vtp/t/collection1/ and leader is http://127.0.0.1:56011/h_vtp/t/collection1/
[junit4:junit4]   2> 3907 T2722 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:56011/h_vtp/t
[junit4:junit4]   2> 3907 T2722 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 3907 T2722 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 3908 T2722 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 3910 T2722 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3911 T2697 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 3912 T2697 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 3912 T2697 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 3926 T2697 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 3935 T2697 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:21162/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@677e2999
[junit4:junit4]   2> 3936 T2697 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 3936 T2724 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:21162. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 3937 T2724 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:21162, initiating session
[junit4:junit4]   2> 3937 T2699 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:14970
[junit4:junit4]   2> 3938 T2699 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:14970
[junit4:junit4]   2> 3939 T2701 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9b66c7020004 with negotiated timeout 10000 for client /140.211.11.196:14970
[junit4:junit4]   2> 3939 T2724 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:21162, sessionid = 0x13c9b66c7020004, negotiated timeout = 10000
[junit4:junit4]   2> 3939 T2725 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@677e2999 name:ZooKeeperConnection Watcher:127.0.0.1:21162/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 3940 T2697 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 3942 T2697 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 3943 T2697 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 4111 T2697 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 4114 T2697 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:30878
[junit4:junit4]   2> 4115 T2697 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 4117 T2697 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 4117 T2697 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1359816873565
[junit4:junit4]   2> 4118 T2697 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1359816873565/solr.xml
[junit4:junit4]   2> 4118 T2697 oasc.CoreContainer.<init> New CoreContainer 1945306197
[junit4:junit4]   2> 4119 T2697 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1359816873565/'
[junit4:junit4]   2> 4119 T2697 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1359816873565/'
[junit4:junit4]   2> 4177 T2697 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 4178 T2697 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 4179 T2697 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 4179 T2697 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 4180 T2697 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 4180 T2697 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 4181 T2697 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 4181 T2697 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 4182 T2697 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 4182 T2697 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 4202 T2697 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 4223 T2697 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:21162/solr
[junit4:junit4]   2> 4223 T2697 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 4224 T2697 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:21162 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@2bcd6006
[junit4:junit4]   2> 4225 T2697 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4226 T2735 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:21162. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4226 T2735 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:21162, initiating session
[junit4:junit4]   2> 4226 T2699 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:65146
[junit4:junit4]   2> 4227 T2699 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:65146
[junit4:junit4]   2> 4228 T2701 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9b66c7020005 with negotiated timeout 20000 for client /140.211.11.196:65146
[junit4:junit4]   2> 4228 T2735 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:21162, sessionid = 0x13c9b66c7020005, negotiated timeout = 20000
[junit4:junit4]   2> 4229 T2736 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2bcd6006 name:ZooKeeperConnection Watcher:127.0.0.1:21162 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4229 T2697 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4231 T2702 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c9b66c7020005
[junit4:junit4]   2> 4232 T2736 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 4232 T2699 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:65146 which had sessionid 0x13c9b66c7020005
[junit4:junit4]   2> 4232 T2697 oaz.ZooKeeper.close Session: 0x13c9b66c7020005 closed
[junit4:junit4]   2> 4233 T2697 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 4244 T2697 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:21162/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@787f92d3
[junit4:junit4]   2> 4245 T2697 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4246 T2737 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:21162. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4246 T2737 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:21162, initiating session
[junit4:junit4]   2> 4246 T2699 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:40852
[junit4:junit4]   2> 4247 T2699 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:40852
[junit4:junit4]   2> 4248 T2701 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9b66c7020006 with negotiated timeout 20000 for client /140.211.11.196:40852
[junit4:junit4]   2> 4248 T2737 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:21162, sessionid = 0x13c9b66c7020006, negotiated timeout = 20000
[junit4:junit4]   2> 4249 T2738 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@787f92d3 name:ZooKeeperConnection Watcher:127.0.0.1:21162/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4249 T2697 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4250 T2702 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9b66c7020006 type:create cxid:0x1 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4252 T2702 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9b66c7020006 type:create cxid:0x2 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4253 T2697 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:30878_h_vtp%2Ft
[junit4:junit4]   2> 4254 T2702 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9b66c7020006 type:delete cxid:0x4 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:30878_h_vtp%2Ft Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:30878_h_vtp%2Ft
[junit4:junit4]   2> 4261 T2697 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:30878_h_vtp%2Ft
[junit4:junit4]   2> 4264 T2719 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 4264 T2725 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 4266 T2719 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 4268 T2697 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 4275 T2739 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1359816873565/collection1
[junit4:junit4]   2> 4275 T2739 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 4276 T2739 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 4276 T2739 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 4277 T2739 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1359816873565/collection1/'
[junit4:junit4]   2> 4279 T2739 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty1-1359816873565/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 4279 T2739 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty1-1359816873565/collection1/lib/README' to classloader
[junit4:junit4]   2> 4336 T2739 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 4411 T2739 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 4512 T2739 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 4519 T2739 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 5169 T2739 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 5180 T2739 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 5184 T2739 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 5197 T2739 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5202 T2739 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5207 T2739 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5208 T2739 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5209 T2739 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty1-1359816873565/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty1/
[junit4:junit4]   2> 5210 T2739 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@12bef803
[junit4:junit4]   2> 5210 T2739 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 5211 T2739 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty1 forceNew:false
[junit4:junit4]   2> 5212 T2739 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty1
[junit4:junit4]   2> 5212 T2739 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty1/index/
[junit4:junit4]   2> 5213 T2739 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 5213 T2739 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty1/index forceNew:false
[junit4:junit4]   2> 5247 T2739 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5227b13d; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 5247 T2739 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 5248 T2739 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty1/index
[junit4:junit4]   2> 5249 T2739 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 5251 T2739 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 5251 T2739 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 5252 T2739 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 5252 T2739 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 5253 T2739 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 5253 T2739 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 5253 T2739 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 5254 T2739 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 5254 T2739 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 5255 T2739 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 5256 T2739 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 5256 T2739 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 5257 T2739 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 5257 T2739 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 5257 T2739 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 5258 T2739 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 5258 T2739 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5259 T2739 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5259 T2739 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5260 T2739 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5261 T2739 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5261 T2739 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5262 T2739 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5262 T2739 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5263 T2739 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 5263 T2739 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 5276 T2739 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 5281 T2739 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 5282 T2739 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 5285 T2739 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty1
[junit4:junit4]   2> 5287 T2739 oass.SolrIndexSearcher.<init> Opening Searcher@6165fee1 main
[junit4:junit4]   2> 5288 T2739 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 5288 T2739 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 5289 T2739 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 5297 T2739 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 5346 T2740 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6165fee1 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 5350 T2739 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 5350 T2739 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 5382 T2720 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 5383 T2720 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:56011_h_vtp%2Ft",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:56011/h_vtp/t"}
[junit4:junit4]   2> 5387 T2720 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:30878_h_vtp%2Ft",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:30878/h_vtp/t"}
[junit4:junit4]   2> 5387 T2720 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 1
[junit4:junit4]   2> 5387 T2720 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 5398 T2725 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5398 T2738 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5398 T2719 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6353 T2739 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 6353 T2739 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:30878/h_vtp/t collection:collection1 shard:shard1
[junit4:junit4]   2> 6354 T2739 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 6401 T2702 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9b66c7020006 type:delete cxid:0x43 zxid:0x5a txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 6402 T2739 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 6403 T2702 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9b66c7020006 type:create cxid:0x44 zxid:0x5b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6411 T2739 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 6412 T2739 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 6412 T2739 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:30878/h_vtp/t/collection1/
[junit4:junit4]   2> 6412 T2739 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 6412 T2739 oasc.SyncStrategy.syncToMe http://127.0.0.1:30878/h_vtp/t/collection1/ has no replicas
[junit4:junit4]   2> 6413 T2739 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:30878/h_vtp/t/collection1/
[junit4:junit4]   2> 6413 T2739 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 6417 T2702 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9b66c7020006 type:create cxid:0x4e zxid:0x5f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6911 T2720 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6930 T2719 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6930 T2725 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6930 T2738 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6941 T2739 oasc.ZkController.register We are http://127.0.0.1:30878/h_vtp/t/collection1/ and leader is http://127.0.0.1:30878/h_vtp/t/collection1/
[junit4:junit4]   2> 6941 T2739 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:30878/h_vtp/t
[junit4:junit4]   2> 6941 T2739 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 6941 T2739 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 6942 T2739 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 6943 T2739 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6944 T2697 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 6945 T2697 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 6945 T2697 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 7120 T2697 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 7123 T2697 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:14367
[junit4:junit4]   2> 7124 T2697 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 7125 T2697 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 7125 T2697 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1359816876580
[junit4:junit4]   2> 7125 T2697 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1359816876580/solr.xml
[junit4:junit4]   2> 7126 T2697 oasc.CoreContainer.<init> New CoreContainer 1740916061
[junit4:junit4]   2> 7127 T2697 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1359816876580/'
[junit4:junit4]   2> 7127 T2697 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1359816876580/'
[junit4:junit4]   2> 7184 T2697 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 7185 T2697 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 7185 T2697 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 7185 T2697 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 7186 T2697 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 7186 T2697 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 7187 T2697 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 7187 T2697 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 7188 T2697 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 7188 T2697 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 7209 T2697 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 7229 T2697 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:21162/solr
[junit4:junit4]   2> 7229 T2697 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 7230 T2697 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:21162 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@1fdb755
[junit4:junit4]   2> 7231 T2697 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 7231 T2750 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:21162. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 7232 T2750 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:21162, initiating session
[junit4:junit4]   2> 7232 T2699 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:60788
[junit4:junit4]   2> 7233 T2699 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:60788
[junit4:junit4]   2> 7234 T2701 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9b66c7020007 with negotiated timeout 20000 for client /140.211.11.196:60788
[junit4:junit4]   2> 7234 T2750 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:21162, sessionid = 0x13c9b66c7020007, negotiated timeout = 20000
[junit4:junit4]   2> 7235 T2751 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1fdb755 name:ZooKeeperConnection Watcher:127.0.0.1:21162 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 7235 T2697 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 7236 T2702 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c9b66c7020007
[junit4:junit4]   2> 7240 T2699 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:60788 which had sessionid 0x13c9b66c7020007
[junit4:junit4]   2> 7240 T2751 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 7240 T2697 oaz.ZooKeeper.close Session: 0x13c9b66c7020007 closed
[junit4:junit4]   2> 7241 T2697 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 7253 T2697 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:21162/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@3b2ccaef
[junit4:junit4]   2> 7254 T2697 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 7254 T2752 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:21162. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 7255 T2752 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:21162, initiating session
[junit4:junit4]   2> 7255 T2699 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:51520
[junit4:junit4]   2> 7256 T2699 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:51520
[junit4:junit4]   2> 7258 T2701 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9b66c7020008 with negotiated timeout 20000 for client /140.211.11.196:51520
[junit4:junit4]   2> 7258 T2752 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:21162, sessionid = 0x13c9b66c7020008, negotiated timeout = 20000
[junit4:junit4]   2> 7258 T2753 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3b2ccaef name:ZooKeeperConnection Watcher:127.0.0.1:21162/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 7258 T2697 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 7260 T2702 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9b66c7020008 type:create cxid:0x1 zxid:0x6c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7261 T2702 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9b66c7020008 type:create cxid:0x2 zxid:0x6d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7273 T2697 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:14367_h_vtp%2Ft
[junit4:junit4]   2> 7274 T2702 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9b66c7020008 type:delete cxid:0x4 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:14367_h_vtp%2Ft Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:14367_h_vtp%2Ft
[junit4:junit4]   2> 7275 T2697 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:14367_h_vtp%2Ft
[junit4:junit4]   2> 7281 T2719 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7281 T2738 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7281 T2725 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7282 T2725 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 7283 T2719 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7283 T2738 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7284 T2697 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 7291 T2754 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1359816876580/collection1
[junit4:junit4]   2> 7291 T2754 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 7292 T2754 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 7292 T2754 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 7293 T2754 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1359816876580/collection1/'
[junit4:junit4]   2> 7295 T2754 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty2-1359816876580/collection1/lib/README' to classloader
[junit4:junit4]   2> 7295 T2754 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty2-1359816876580/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 7350 T2754 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 7423 T2754 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 7524 T2754 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 7530 T2754 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 8442 T2720 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8442 T2720 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:30878_h_vtp%2Ft",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:30878/h_vtp/t"}
[junit4:junit4]   2> 8455 T2754 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 8455 T2719 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8455 T2753 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8455 T2725 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8455 T2738 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8468 T2754 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 8472 T2754 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 8485 T2754 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8490 T2754 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8495 T2754 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8496 T2754 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8497 T2754 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty2-1359816876580/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty2/
[junit4:junit4]   2> 8498 T2754 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@12bef803
[junit4:junit4]   2> 8498 T2754 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 8499 T2754 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty2 forceNew:false
[junit4:junit4]   2> 8500 T2754 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty2
[junit4:junit4]   2> 8500 T2754 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty2/index/
[junit4:junit4]   2> 8501 T2754 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 8501 T2754 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty2/index forceNew:false
[junit4:junit4]   2> 8508 T2754 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@443844c7; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 8508 T2754 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 8509 T2754 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty2/index
[junit4:junit4]   2> 8510 T2754 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 8512 T2754 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 8512 T2754 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 8512 T2754 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 8513 T2754 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 8513 T2754 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 8514 T2754 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 8514 T2754 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 8515 T2754 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 8515 T2754 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 8516 T2754 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 8516 T2754 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 8517 T2754 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 8517 T2754 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 8518 T2754 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 8518 T2754 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 8519 T2754 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 8519 T2754 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8520 T2754 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8520 T2754 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8521 T2754 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8521 T2754 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8522 T2754 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8523 T2754 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8523 T2754 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8524 T2754 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 8524 T2754 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 8537 T2754 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 8542 T2754 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 8543 T2754 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 8546 T2754 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty2
[junit4:junit4]   2> 8547 T2754 oass.SolrIndexSearcher.<init> Opening Searcher@5e961fa9 main
[junit4:junit4]   2> 8548 T2754 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 8549 T2754 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 8549 T2754 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 8556 T2754 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 8615 T2755 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5e961fa9 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 8620 T2754 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 8620 T2754 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 9960 T2720 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9961 T2720 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:14367_h_vtp%2Ft",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:14367/h_vtp/t"}
[junit4:junit4]   2> 9962 T2720 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4]   2> 9962 T2720 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 9975 T2719 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9975 T2725 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9975 T2738 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9975 T2753 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10624 T2754 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 10624 T2754 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:14367/h_vtp/t collection:collection1 shard:shard1
[junit4:junit4]   2> 10627 T2754 oasc.ZkController.register We are http://127.0.0.1:14367/h_vtp/t/collection1/ and leader is http://127.0.0.1:30878/h_vtp/t/collection1/
[junit4:junit4]   2> 10627 T2754 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:14367/h_vtp/t
[junit4:junit4]   2> 10628 T2754 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 10628 T2754 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C213 name=collection1 org.apache.solr.core.SolrCore@60b0c0ba url=http://127.0.0.1:14367/h_vtp/t/collection1 node=127.0.0.1:14367_h_vtp%2Ft C213_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:14367_h_vtp%2Ft, base_url=http://127.0.0.1:14367/h_vtp/t}
[junit4:junit4]   2> 10628 T2756 C213 P14367 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 10629 T2754 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 10630 T2756 C213 P14367 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 10630 T2756 C213 P14367 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 10630 T2756 C213 P14367 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 10631 T2697 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 10631 T2697 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 10632 T2756 C213 P14367 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 10632 T2697 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 10813 T2697 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 10817 T2697 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:14607
[junit4:junit4]   2> 10817 T2697 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 10818 T2697 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 10818 T2697 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1359816880269
[junit4:junit4]   2> 10819 T2697 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1359816880269/solr.xml
[junit4:junit4]   2> 10819 T2697 oasc.CoreContainer.<init> New CoreContainer 260397513
[junit4:junit4]   2> 10820 T2697 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1359816880269/'
[junit4:junit4]   2> 10821 T2697 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1359816880269/'
[junit4:junit4]   2> 10878 T2697 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 10878 T2697 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 10879 T2697 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 10879 T2697 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 10880 T2697 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 10880 T2697 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 10881 T2697 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 10881 T2697 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 10882 T2697 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 10882 T2697 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 10903 T2697 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 10924 T2697 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:21162/solr
[junit4:junit4]   2> 10924 T2697 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 10925 T2697 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:21162 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@45a195c5
[junit4:junit4]   2> 10926 T2697 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 10926 T2766 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:21162. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 10927 T2766 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:21162, initiating session
[junit4:junit4]   2> 10927 T2699 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:43838
[junit4:junit4]   2> 10928 T2699 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:43838
[junit4:junit4]   2> 10930 T2701 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9b66c7020009 with negotiated timeout 20000 for client /140.211.11.196:43838
[junit4:junit4]   2> 10930 T2766 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:21162, sessionid = 0x13c9b66c7020009, negotiated timeout = 20000
[junit4:junit4]   2> 10930 T2767 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@45a195c5 name:ZooKeeperConnection Watcher:127.0.0.1:21162 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 10931 T2697 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 10932 T2702 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c9b66c7020009
[junit4:junit4]   2> 10944 T2767 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 10944 T2699 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:43838 which had sessionid 0x13c9b66c7020009
[junit4:junit4]   2> 10944 T2697 oaz.ZooKeeper.close Session: 0x13c9b66c7020009 closed
[junit4:junit4]   2> 10945 T2697 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 10955 T2697 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:21162/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@794a038b
[junit4:junit4]   2> 10956 T2697 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 10956 T2768 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:21162. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 10957 T2768 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:21162, initiating session
[junit4:junit4]   2> 10957 T2699 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:65365
[junit4:junit4]   2> 10957 T2699 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:65365
[junit4:junit4]   2> 10959 T2701 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9b66c702000a with negotiated timeout 20000 for client /140.211.11.196:65365
[junit4:junit4]   2> 10959 T2768 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:21162, sessionid = 0x13c9b66c702000a, negotiated timeout = 20000
[junit4:junit4]   2> 10959 T2769 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@794a038b name:ZooKeeperConnection Watcher:127.0.0.1:21162/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 10959 T2697 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 10960 T2702 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9b66c702000a type:create cxid:0x1 zxid:0x7f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 10962 T2702 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9b66c702000a type:create cxid:0x2 zxid:0x80 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 10964 T2697 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:14607_h_vtp%2Ft
[junit4:junit4]   2> 10964 T2702 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9b66c702000a type:delete cxid:0x4 zxid:0x81 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:14607_h_vtp%2Ft Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:14607_h_vtp%2Ft
[junit4:junit4]   2> 10966 T2697 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:14607_h_vtp%2Ft
[junit4:junit4]   2> 10973 T2738 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10973 T2719 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10973 T2753 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10974 T2725 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 10974 T2725 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 10975 T2738 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 10976 T2719 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 10976 T2753 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 10977 T2697 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 10984 T2770 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1359816880269/collection1
[junit4:junit4]   2> 10984 T2770 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 10985 T2770 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 10985 T2770 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 10987 T2770 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1359816880269/collection1/'
[junit4:junit4]   2> 10988 T2770 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty3-1359816880269/collection1/lib/README' to classloader
[junit4:junit4]   2> 10988 T2770 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty3-1359816880269/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 11049 T2770 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 11126 T2770 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 11227 T2770 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 11233 T2770 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 11479 T2720 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11480 T2720 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:14367_h_vtp%2Ft",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:14367/h_vtp/t"}
[junit4:junit4]   2> 11526 T2719 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11526 T2769 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11526 T2753 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11526 T2725 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11526 T2738 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11892 T2770 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 11903 T2770 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 11907 T2770 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 11921 T2770 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 11925 T2770 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 11930 T2770 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 11931 T2770 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 11932 T2770 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty3-1359816880269/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty3/
[junit4:junit4]   2> 11932 T2770 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@12bef803
[junit4:junit4]   2> 11933 T2770 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 11934 T2770 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty3 forceNew:false
[junit4:junit4]   2> 11934 T2770 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty3
[junit4:junit4]   2> 11935 T2770 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty3/index/
[junit4:junit4]   2> 11935 T2770 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 11936 T2770 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty3/index forceNew:false
[junit4:junit4]   2> 11942 T2770 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@710bc4a7; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 11942 T2770 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 11943 T2770 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty3/index
[junit4:junit4]   2> 11944 T2770 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 11946 T2770 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 11946 T2770 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 11946 T2770 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 11947 T2770 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 11947 T2770 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 11948 T2770 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 11948 T2770 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 11948 T2770 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 11949 T2770 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 11950 T2770 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 11950 T2770 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 11951 T2770 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 11951 T2770 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 11951 T2770 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 11952 T2770 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 11952 T2770 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 11953 T2770 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11953 T2770 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11954 T2770 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11954 T2770 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11955 T2770 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11955 T2770 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11956 T2770 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11956 T2770 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11957 T2770 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 11957 T2770 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 11969 T2770 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 11973 T2770 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 11974 T2770 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 11977 T2770 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty3
[junit4:junit4]   2> 11978 T2770 oass.SolrIndexSearcher.<init> Opening Searcher@39ebca1c main
[junit4:junit4]   2> 11979 T2770 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 11980 T2770 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 11980 T2770 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 11987 T2770 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 12046 T2771 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@39ebca1c main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 12052 T2770 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 12052 T2770 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 13032 T2720 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 13033 T2720 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:14607_h_vtp%2Ft",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:14607/h_vtp/t"}
[junit4:junit4]   2> 13034 T2720 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4]   2> 13034 T2720 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 13047 T2719 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13047 T2769 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13047 T2738 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13047 T2725 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13047 T2753 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13054 T2770 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 13054 T2770 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:14607/h_vtp/t collection:collection1 shard:shard1
[junit4:junit4]   2> 13057 T2770 oasc.ZkController.register We are http://127.0.0.1:14607/h_vtp/t/collection1/ and leader is http://127.0.0.1:30878/h_vtp/t/collection1/
[junit4:junit4]   2> 13057 T2770 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:14607/h_vtp/t
[junit4:junit4]   2> 13058 T2770 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 13058 T2770 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C214 name=collection1 org.apache.solr.core.SolrCore@5d4854e8 url=http://127.0.0.1:14607/h_vtp/t/collection1 node=127.0.0.1:14607_h_vtp%2Ft C214_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:14607_h_vtp%2Ft, base_url=http://127.0.0.1:14607/h_vtp/t}
[junit4:junit4]   2> 13058 T2772 C214 P14607 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 13059 T2770 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 13060 T2772 C214 P14607 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 13060 T2772 C214 P14607 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 13060 T2772 C214 P14607 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 13061 T2697 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 13061 T2697 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 13062 T2772 C214 P14607 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 13062 T2697 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 13267 T2697 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 13269 T2697 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:26299
[junit4:junit4]   2> 13270 T2697 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 13271 T2697 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 13271 T2697 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1359816882703
[junit4:junit4]   2> 13272 T2697 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1359816882703/solr.xml
[junit4:junit4]   2> 13272 T2697 oasc.CoreContainer.<init> New CoreContainer 1331693958
[junit4:junit4]   2> 13273 T2697 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1359816882703/'
[junit4:junit4]   2> 13274 T2697 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1359816882703/'
[junit4:junit4]   2> 13330 T2697 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 13331 T2697 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 13331 T2697 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 13332 T2697 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 13332 T2697 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 13333 T2697 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 13333 T2697 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 13334 T2697 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 13334 T2697 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 13335 T2697 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 13355 T2697 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 13375 T2697 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:21162/solr
[junit4:junit4]   2> 13376 T2697 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 13377 T2697 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:21162 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@41a7bec1
[junit4:junit4]   2> 13378 T2697 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 13378 T2782 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:21162. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 13379 T2782 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:21162, initiating session
[junit4:junit4]   2> 13379 T2699 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:33281
[junit4:junit4]   2> 13380 T2699 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:33281
[junit4:junit4]   2> 13382 T2701 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9b66c702000b with negotiated timeout 20000 for client /140.211.11.196:33281
[junit4:junit4]   2> 13382 T2782 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:21162, sessionid = 0x13c9b66c702000b, negotiated timeout = 20000
[junit4:junit4]   2> 13382 T2783 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@41a7bec1 name:ZooKeeperConnection Watcher:127.0.0.1:21162 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 13382 T2697 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 13385 T2702 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c9b66c702000b
[junit4:junit4]   2> 13394 T2783 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 13394 T2699 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:33281 which had sessionid 0x13c9b66c702000b
[junit4:junit4]   2> 13394 T2697 oaz.ZooKeeper.close Session: 0x13c9b66c702000b closed
[junit4:junit4]   2> 13395 T2697 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 13404 T2697 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:21162/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@25d56f62
[junit4:junit4]   2> 13406 T2697 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 13406 T2784 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:21162. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 13406 T2784 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:21162, initiating session
[junit4:junit4]   2> 13407 T2699 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:13832
[junit4:junit4]   2> 13407 T2699 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:13832
[junit4:junit4]   2> 13408 T2701 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c9b66c702000c with negotiated timeout 20000 for client /140.211.11.196:13832
[junit4:junit4]   2> 13409 T2784 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:21162, sessionid = 0x13c9b66c702000c, negotiated timeout = 20000
[junit4:junit4]   2> 13409 T2785 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@25d56f62 name:ZooKeeperConnection Watcher:127.0.0.1:21162/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 13409 T2697 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 13410 T2702 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9b66c702000c type:create cxid:0x1 zxid:0x92 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 13412 T2702 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9b66c702000c type:create cxid:0x2 zxid:0x93 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 13422 T2697 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:26299_h_vtp%2Ft
[junit4:junit4]   2> 13423 T2702 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c9b66c702000c type:delete cxid:0x4 zxid:0x94 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:26299_h_vtp%2Ft Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:26299_h_vtp%2Ft
[junit4:junit4]   2> 13424 T2697 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:26299_h_vtp%2Ft
[junit4:junit4]   2> 13431 T2719 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13431 T2738 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13431 T2769 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13431 T2753 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13431 T2725 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 13431 T2725 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 13433 T2719 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 13433 T2753 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 13434 T2769 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 13434 T2738 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 13435 T2697 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 13442 T2786 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1359816882703/collection1
[junit4:junit4]   2> 13442 T2786 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 13443 T2786 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 13443 T2786 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 13444 T2786 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1359816882703/collection1/'
[junit4:junit4]   2> 13446 T2786 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty4-1359816882703/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 13446 T2786 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-jetty4-1359816882703/collection1/lib/README' to classloader
[junit4:junit4]   2> 13501 T2786 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 13574 T2786 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2>  C213_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:14367_h_vtp%2Ft, base_url=http://127.0.0.1:14367/h_vtp/t}
[junit4:junit4]   2> 13652 T2756 C213 P14367 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:30878/h_vtp/t/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 13652 T2756 C213 P14367 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:14367/h_vtp/t START replicas=[http://127.0.0.1:30878/h_vtp/t/collection1/] nUpdates=100
[junit4:junit4]   2> 13652 T2756 C213 P14367 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 13653 T2756 C213 P14367 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 13653 T2756 C213 P14367 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 13653 T2756 C213 P14367 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 13653 T2756 C213 P14367 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 13653 T2756 C213 P14367 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:30878/h_vtp/t/collection1/. core=collection1
[junit4:junit4]   2> 13654 T2756 C213 P14367 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C215 name=collection1 org.apache.solr.core.SolrCore@1d080c9a url=http://127.0.0.1:30878/h_vtp/t/collection1 node=127.0.0.1:30878_h_vtp%2Ft C215_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:30878_h_vtp%2Ft, base_url=http://127.0.0.1:30878/h_vtp/t, leader=true}
[junit4:junit4]   2> 13661 T2730 C215 P30878 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 13665 T2731 C215 P30878 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 13666 T2731 C215 P30878 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty1
[junit4:junit4]   2> 13670 T2731 C215 P30878 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5227b13d; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 13671 T2731 C215 P30878 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 13675 T2786 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 13691 T2786 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 13703 T2731 C215 P30878 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5227b13d; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5227b13d; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 13704 T2731 C215 P30878 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 13706 T2731 C215 P30878 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty1
[junit4:junit4]   2> 13707 T2731 C215 P30878 oass.SolrIndexSearcher.<init> Opening Searcher@74df71b7 realtime
[junit4:junit4]   2> 13707 T2731 C215 P30878 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 13708 T2731 C215 P30878 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 43
[junit4:junit4]   2> 13709 T2756 C213 P14367 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 13710 T2756 C213 P14367 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 13713 T2732 C215 P30878 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 13713 T2732 C215 P30878 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 13714 T2756 C213 P14367 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 13714 T2756 C213 P14367 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 13714 T2756 C213 P14367 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 13716 T2732 C215 P30878 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty1
[junit4:junit4]   2> 13717 T2732 C215 P30878 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty1/index
[junit4:junit4]   2> 13718 T2732 C215 P30878 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=2 
[junit4:junit4]   2> 13718 T2756 C213 P14367 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 13720 T2756 C213 P14367 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty2/index.20130203015443340 forceNew:false
[junit4:junit4]   2> 13720 T2756 C213 P14367 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty2
[junit4:junit4]   2> 13721 T2756 C213 P14367 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty2/index.20130203015443340 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6fc05777; maxCacheMB=48.0 maxMergeSizeMB=4.0)) fullCopy=true
[junit4:junit4]   2> 13725 T2732 C215 P30878 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> 13727 T2756 C213 P14367 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty2/index
[junit4:junit4]   2> 13728 T2756 C213 P14367 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 13728 T2756 C213 P14367 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130203015443340
[junit4:junit4]   2> 13730 T2756 C213 P14367 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty2
[junit4:junit4]   2> 13730 T2756 C213 P14367 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty2
[junit4:junit4]   2> 13731 T2756 C213 P14367 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty2/index/ new=./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty2/index.20130203015443340
[junit4:junit4]   2> 13735 T2756 C213 P14367 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty2/index.20130203015443340 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6fc05777; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 13735 T2756 C213 P14367 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 13737 T2756 C213 P14367 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty2
[junit4:junit4]   2> 13737 T2756 C213 P14367 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 13737 T2756 C213 P14367 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 13737 T2756 C213 P14367 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 13738 T2756 C213 P14367 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty2/index.20130203015443340
[junit4:junit4]   2> 13739 T2756 C213 P14367 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty2
[junit4:junit4]   2> 13743 T2756 C213 P14367 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty2/index.20130203015443340 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6fc05777; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 13744 T2756 C213 P14367 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 13744 T2756 C213 P14367 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 13745 T2756 C213 P14367 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty2
[junit4:junit4]   2> 13745 T2756 C213 P14367 oass.SolrIndexSearcher.<init> Opening Searcher@4201ba20 main
[junit4:junit4]   2> 13747 T2755 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4201ba20 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 13747 T2755 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty2/index
[junit4:junit4]   2> 13747 T2756 C213 P14367 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 13751 T2756 C213 P14367 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty2/index.20130203015443340 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6fc05777; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty2/index.20130203015443340 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6fc05777; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 13752 T2756 C213 P14367 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 13753 T2756 C213 P14367 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty2
[junit4:junit4]   2> 13753 T2756 C213 P14367 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 13753 T2756 C213 P14367 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty2/index.20130203015443340
[junit4:junit4]   2> 13754 T2756 C213 P14367 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty2/index
[junit4:junit4]   2> 13754 T2756 C213 P14367 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 13754 T2756 C213 P14367 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 13754 T2756 C213 P14367 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 13755 T2756 C213 P14367 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 13761 T2756 C213 P14367 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 14319 T2786 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 14330 T2786 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 14335 T2786 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 14348 T2786 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 14365 T2786 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 14370 T2786 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 14371 T2786 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 14373 T2786 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-jetty4-1359816882703/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty4/
[junit4:junit4]   2> 14373 T2786 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@12bef803
[junit4:junit4]   2> 14373 T2786 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 14374 T2786 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty4 forceNew:false
[junit4:junit4]   2> 14375 T2786 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty4
[junit4:junit4]   2> 14375 T2786 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty4/index/
[junit4:junit4]   2> 14376 T2786 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 14377 T2786 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty4/index forceNew:false
[junit4:junit4]   2> 14383 T2786 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6f99d33c; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 14383 T2786 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 14385 T2786 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty4/index
[junit4:junit4]   2> 14385 T2786 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 14387 T2786 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 14387 T2786 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 14388 T2786 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 14388 T2786 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 14389 T2786 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 14389 T2786 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 14389 T2786 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 14390 T2786 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 14391 T2786 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 14391 T2786 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 14392 T2786 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 14392 T2786 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 14393 T2786 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 14393 T2786 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 14394 T2786 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 14394 T2786 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 14395 T2786 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14395 T2786 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14396 T2786 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14396 T2786 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14397 T2786 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14397 T2786 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14398 T2786 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14398 T2786 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14399 T2786 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 14400 T2786 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 14414 T2786 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 14419 T2786 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 14421 T2786 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 14423 T2786 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty4
[junit4:junit4]   2> 14425 T2786 oass.SolrIndexSearcher.<init> Opening Searcher@542bf1d0 main
[junit4:junit4]   2> 14426 T2786 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 14426 T2786 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 14427 T2786 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 14442 T2786 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 14525 T2789 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@542bf1d0 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 14530 T2786 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 14531 T2786 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 14551 T2720 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14552 T2720 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:14607_h_vtp%2Ft",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:14607/h_vtp/t"}
[junit4:junit4]   2> 14558 T2720 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:14367_h_vtp%2Ft",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:14367/h_vtp/t"}
[junit4:junit4]   2> 14572 T2720 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:26299_h_vtp%2Ft",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:26299/h_vtp/t"}
[junit4:junit4]   2> 14572 T2720 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4]   2> 14572 T2720 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 14584 T2719 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14584 T2769 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14584 T2725 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14584 T2738 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14584 T2753 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14584 T2785 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15538 T2786 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 15538 T2786 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:26299/h_vtp/t collection:collection1 shard:shard1
[junit4:junit4]   2> 15542 T2786 oasc.ZkController.register We are http://127.0.0.1:26299/h_vtp/t/collection1/ and leader is http://127.0.0.1:30878/h_vtp/t/collection1/
[junit4:junit4]   2> 15542 T2786 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:26299/h_vtp/t
[junit4:junit4]   2> 15542 T2786 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 15543 T2786 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C216 name=collection1 org.apache.solr.core.SolrCore@5fe82e4d url=http://127.0.0.1:26299/h_vtp/t/collection1 node=127.0.0.1:26299_h_vtp%2Ft C216_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:26299_h_vtp%2Ft, base_url=http://127.0.0.1:26299/h_vtp/t}
[junit4:junit4]   2> 15544 T2790 C216 P26299 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 15544 T2790 C216 P26299 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 15544 T2786 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 15545 T2790 C216 P26299 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 15545 T2790 C216 P26299 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 15546 T2697 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 15546 T2697 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 15547 T2790 C216 P26299 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 15547 T2697 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 15566 T2697 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 15567 T2697 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4]   2> 15568 T2697 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4]   2> 15569 T2697 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16099 T2720 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16100 T2720 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:26299_h_vtp%2Ft",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:26299/h_vtp/t"}
[junit4:junit4]   2> 16199 T2785 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16199 T2725 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16199 T2719 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16199 T2769 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16199 T2753 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16199 T2738 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16570 T2697 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C214_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:14607_h_vtp%2Ft, base_url=http://127.0.0.1:14607/h_vtp/t}
[junit4:junit4]   2> 17086 T2772 C214 P14607 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:30878/h_vtp/t/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 17086 T2772 C214 P14607 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:14607/h_vtp/t START replicas=[http://127.0.0.1:30878/h_vtp/t/collection1/] nUpdates=100
[junit4:junit4]   2> 17088 T2772 C214 P14607 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 17088 T2772 C214 P14607 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 17088 T2772 C214 P14607 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 17088 T2772 C214 P14607 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 17088 T2772 C214 P14607 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 17089 T2772 C214 P14607 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:30878/h_vtp/t/collection1/. core=collection1
[junit4:junit4]   2> 17089 T2772 C214 P14607 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 17090 T2730 C215 P30878 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 17100 T2734 C215 P30878 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 17106 T2734 C215 P30878 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5227b13d; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5227b13d; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 17107 T2734 C215 P30878 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 17107 T2734 C215 P30878 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty1
[junit4:junit4]   2> 17108 T2734 C215 P30878 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 17108 T2734 C215 P30878 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 9
[junit4:junit4]   2> 17109 T2772 C214 P14607 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 17110 T2772 C214 P14607 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 17112 T2732 C215 P30878 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 17113 T2772 C214 P14607 oash.SnapPuller.fetchLatestIndex Master's generation: 3
[junit4:junit4]   2> 17113 T2772 C214 P14607 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 17113 T2772 C214 P14607 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 17116 T2732 C215 P30878 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty1
[junit4:junit4]   2> 17117 T2732 C215 P30878 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty1/index
[junit4:junit4]   2> 17117 T2732 C215 P30878 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=3&version=2} status=0 QTime=1 
[junit4:junit4]   2> 17118 T2772 C214 P14607 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 17120 T2772 C214 P14607 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty3/index.20130203015446740 forceNew:false
[junit4:junit4]   2> 17120 T2772 C214 P14607 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty3
[junit4:junit4]   2> 17121 T2772 C214 P14607 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty3/index.20130203015446740 lockFactory=org.apache.lucene.store.NativeFSLockFactory@56d5e4fe; maxCacheMB=48.0 maxMergeSizeMB=4.0)) fullCopy=true
[junit4:junit4]   2> 17149 T2732 C215 P30878 REQ /replication {file=segments_3&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=3} status=0 QTime=0 
[junit4:junit4]   2> 17152 T2772 C214 P14607 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty3/index
[junit4:junit4]   2> 17152 T2772 C214 P14607 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 17153 T2772 C214 P14607 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130203015446740
[junit4:junit4]   2> 17154 T2772 C214 P14607 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty3
[junit4:junit4]   2> 17155 T2772 C214 P14607 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty3
[junit4:junit4]   2> 17155 T2772 C214 P14607 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty3/index/ new=./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty3/index.20130203015446740
[junit4:junit4]   2> 17159 T2772 C214 P14607 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty3/index.20130203015446740 lockFactory=org.apache.lucene.store.NativeFSLockFactory@56d5e4fe; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 17160 T2772 C214 P14607 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 17161 T2772 C214 P14607 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty3
[junit4:junit4]   2> 17161 T2772 C214 P14607 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 17162 T2772 C214 P14607 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 17162 T2772 C214 P14607 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 17163 T2772 C214 P14607 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty3/index.20130203015446740
[junit4:junit4]   2> 17163 T2772 C214 P14607 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty3
[junit4:junit4]   2> 17167 T2772 C214 P14607 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty3/index.20130203015446740 lockFactory=org.apache.lucene.store.NativeFSLockFactory@56d5e4fe; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 17167 T2772 C214 P14607 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 17167 T2772 C214 P14607 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 17168 T2772 C214 P14607 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty3
[junit4:junit4]   2> 17168 T2772 C214 P14607 oass.SolrIndexSearcher.<init> Opening Searcher@2935183b main
[junit4:junit4]   2> 17170 T2771 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2935183b main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4]   2> 17170 T2771 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty3/index
[junit4:junit4]   2> 17171 T2772 C214 P14607 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 17174 T2772 C214 P14607 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty3/index.20130203015446740 lockFactory=org.apache.lucene.store.NativeFSLockFactory@56d5e4fe; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty3/index.20130203015446740 lockFactory=org.apache.lucene.store.NativeFSLockFactory@56d5e4fe; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_4,generation=4,filenames=[segments_4]
[junit4:junit4]   2> 17174 T2772 C214 P14607 oasc.SolrDeletionPolicy.updateCommits newest commit = 4[segments_4]
[junit4:junit4]   2> 17175 T2772 C214 P14607 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty3
[junit4:junit4]   2> 17175 T2772 C214 P14607 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 17176 T2772 C214 P14607 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty3/index.20130203015446740
[junit4:junit4]   2> 17176 T2772 C214 P14607 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty3/index
[junit4:junit4]   2> 17176 T2772 C214 P14607 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 17176 T2772 C214 P14607 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 17177 T2772 C214 P14607 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 17177 T2772 C214 P14607 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 17184 T2772 C214 P14607 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 17572 T2697 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17726 T2720 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17727 T2720 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:14607_h_vtp%2Ft",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:14607/h_vtp/t"}
[junit4:junit4]   2> 17743 T2719 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17743 T2785 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17743 T2769 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17743 T2725 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17743 T2753 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17743 T2738 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2>  C216_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:26299_h_vtp%2Ft, base_url=http://127.0.0.1:26299/h_vtp/t}
[junit4:junit4]   2> 18569 T2790 C216 P26299 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:30878/h_vtp/t/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 18569 T2790 C216 P26299 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:26299/h_vtp/t START replicas=[http://127.0.0.1:30878/h_vtp/t/collection1/] nUpdates=100
[junit4:junit4]   2> 18571 T2790 C216 P26299 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 18571 T2790 C216 P26299 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 18571 T2790 C216 P26299 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 18571 T2790 C216 P26299 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 18571 T2790 C216 P26299 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 18572 T2790 C216 P26299 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:30878/h_vtp/t/collection1/. core=collection1
[junit4:junit4]   2> 18572 T2790 C216 P26299 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 18573 T2730 C215 P30878 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 18575 T2697 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 18587 T2728 C215 P30878 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 18618 T2728 C215 P30878 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=3
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5227b13d; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5227b13d; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5227b13d; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_4,generation=4,filenames=[segments_4]
[junit4:junit4]   2> 18618 T2728 C215 P30878 oasc.SolrDeletionPolicy.updateCommits newest commit = 4[segments_4]
[junit4:junit4]   2> 18619 T2728 C215 P30878 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty1
[junit4:junit4]   2> 18620 T2728 C215 P30878 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 18620 T2728 C215 P30878 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 33
[junit4:junit4]   2> 18621 T2790 C216 P26299 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 18621 T2790 C216 P26299 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 18624 T2732 C215 P30878 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 18625 T2790 C216 P26299 oash.SnapPuller.fetchLatestIndex Master's generation: 4
[junit4:junit4]   2> 18625 T2790 C216 P26299 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 18625 T2790 C216 P26299 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 18628 T2732 C215 P30878 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty1
[junit4:junit4]   2> 18629 T2732 C215 P30878 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty1/index
[junit4:junit4]   2> 18629 T2732 C215 P30878 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=4&version=2} status=0 QTime=1 
[junit4:junit4]   2> 18630 T2790 C216 P26299 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 18632 T2790 C216 P26299 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty4/index.20130203015448251 forceNew:false
[junit4:junit4]   2> 18632 T2790 C216 P26299 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty4
[junit4:junit4]   2> 18633 T2790 C216 P26299 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty4/index.20130203015448251 lockFactory=org.apache.lucene.store.NativeFSLockFactory@226a54e1; maxCacheMB=48.0 maxMergeSizeMB=4.0)) fullCopy=true
[junit4:junit4]   2> 18637 T2732 C215 P30878 REQ /replication {file=segments_4&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=4} status=0 QTime=0 
[junit4:junit4]   2> 18639 T2790 C216 P26299 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty4/index
[junit4:junit4]   2> 18640 T2790 C216 P26299 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 18640 T2790 C216 P26299 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130203015448251
[junit4:junit4]   2> 18641 T2790 C216 P26299 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty4
[junit4:junit4]   2> 18642 T2790 C216 P26299 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty4
[junit4:junit4]   2> 18642 T2790 C216 P26299 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty4/index/ new=./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty4/index.20130203015448251
[junit4:junit4]   2> 18646 T2790 C216 P26299 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty4/index.20130203015448251 lockFactory=org.apache.lucene.store.NativeFSLockFactory@226a54e1; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_4,generation=4,filenames=[segments_4]
[junit4:junit4]   2> 18646 T2790 C216 P26299 oasc.SolrDeletionPolicy.updateCommits newest commit = 4[segments_4]
[junit4:junit4]   2> 18648 T2790 C216 P26299 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.SyncSliceTest-1359816869627/jetty4
[junit4:junit4]   2> 18648 T2790 C216 P26299 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 18648 T2790 C216 P26299 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 18648 T2790 C216 P26299 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 18649 T2790 C216 P26299 oasc.CachingDirectoryFactory.

[...truncated too long message...]

 > 	at org.junit.Assert.fail(Assert.java:93)
[junit4:junit4]    > 	at org.junit.Assert.assertTrue(Assert.java:43)
[junit4:junit4]    > 	at org.junit.Assert.assertNotNull(Assert.java:526)
[junit4:junit4]    > 	at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:214)
[junit4:junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:794)
[junit4:junit4]    > 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4:junit4]    > 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[junit4:junit4]    > 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4:junit4]    > 	at java.lang.reflect.Method.invoke(Method.java:601)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4:junit4]    > 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]    > 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]    > 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 145466 T2697 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4]   2> NOTE: test params are: codec=Lucene42: {range_facet_l=MockVariableIntBlock(baseBlockSize=85), text=PostingsFormat(name=Lucene41WithOrds), _version_=MockVariableIntBlock(baseBlockSize=85), rnd_b=PostingsFormat(name=TestBloomFilteredLucene41Postings), intDefault=MockVariableIntBlock(baseBlockSize=85), timestamp=MockVariableIntBlock(baseBlockSize=85), id=PostingsFormat(name=TestBloomFilteredLucene41Postings), a_t=MockVariableIntBlock(baseBlockSize=85), range_facet_sl=PostingsFormat(name=Lucene41WithOrds), range_facet_si=PostingsFormat(name=Asserting), other_tl1=MockVariableIntBlock(baseBlockSize=85), multiDefault=PostingsFormat(name=Asserting), a_si=PostingsFormat(name=Asserting)}, sim=DefaultSimilarity, locale=vi, timezone=Australia/Sydney
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=2,free=255532880,total=534773760
[junit4:junit4]   2> NOTE: All tests run in this JVM: [TestDFRSimilarityFactory, TestCodecSupport, TestTrie, OutputWriterTest, TestRemoteStreaming, TestDistributedGrouping, TestRealTimeGet, ConvertedLegacyTest, SchemaVersionSpecificBehaviorTest, TestArbitraryIndexDir, RegexBoostProcessorTest, DOMUtilTest, BasicZkTest, SynonymTokenizerTest, TestMultiCoreConfBootstrap, SuggesterTSTTest, BadIndexSchemaTest, DocumentAnalysisRequestHandlerTest, TestDocSet, TestPropInject, TestSolrJ, QueryEqualityTest, ShardRoutingTest, PolyFieldTest, FileUtilsTest, DirectSolrSpellCheckerTest, IndexSchemaTest, EchoParamsTest, TestStressLucene, TestSolrDeletionPolicy1, TestLRUCache, TestWriterPerf, UUIDFieldTest, BasicFunctionalityTest, ClusterStateTest, XsltUpdateRequestHandlerTest, RequiredFieldsTest, DirectUpdateHandlerTest, TermVectorComponentTest, DistributedQueryElevationComponentTest, StandardRequestHandlerTest, ShowFileRequestHandlerTest, PrimitiveFieldTypeTest, TestReplicationHandler, TestRTGBase, TestJmxIntegration, SoftAutoCommitTest, PathHierarchyTokenizerFactoryTest, ShardRoutingCustomTest, PrimUtilsTest, TestSolrDeletionPolicy2, TestUpdate, LegacyHTMLStripCharFilterTest, TestCSVResponseWriter, TermVectorComponentDistributedTest, TestStressReorder, TestOmitPositions, HighlighterTest, LukeRequestHandlerTest, TestSurroundQueryParser, BasicDistributedZkTest, ZkSolrClientTest, TestHashPartitioner, StatsComponentTest, TestRecovery, SolrCoreTest, TestRangeQuery, SimpleFacetsTest, TestCoreContainer, TestSort, TestGroupingSearch, PeerSyncTest, SignatureUpdateProcessorFactoryTest, SampleTest, CoreAdminHandlerTest, TestFunctionQuery, TestCSVLoader, MBeansHandlerTest, DisMaxRequestHandlerTest, QueryParsingTest, MinimalSchemaTest, XmlUpdateRequestHandlerTest, JsonLoaderTest, IndexSchemaRuntimeFieldTest, TestValueSourceCache, TestReversedWildcardFilterFactory, DebugComponentTest, ReturnFieldsTest, SearchHandlerTest, TestQueryUtils, UpdateParamsTest, CSVRequestHandlerTest, TestSearchPerf, AlternateDirectoryTest, FieldMutatingUpdateProcessorTest, TestPropInjectDefaults, TestXIncludeConfig, TestQuerySenderListener, SolrIndexConfigTest, NumericFieldsTest, TestSolrCoreProperties, TestPhraseSuggestions, TestPluginEnable, TestLMJelinekMercerSimilarityFactory, TimeZoneUtilsTest, DateMathParserTest, PluginInfoTest, OpenExchangeRatesOrgProviderTest, TestSolrXMLSerializer, TestSuggestSpellingConverter, ChaosMonkeySafeLeaderTest, SystemInfoHandlerTest, ChaosMonkeyNothingIsSafeTest, TestDocumentBuilder, ClusterStateUpdateTest, CollectionsAPIDistributedZkTest, OverseerCollectionProcessorTest, SyncSliceTest]
[junit4:junit4] Completed on J1 in 145.49s, 1 test, 1 failure <<< FAILURES!

[...truncated 10 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:346: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/common-build.xml:428: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:1176: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:841: There were test failures: 249 suites, 1038 tests, 1 failure, 23 ignored (5 assumptions)

Total time: 41 minutes 49 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-trunk-java7 - Build # 3701 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-java7/3701/

1 tests failed.
FAILED:  org.apache.solr.cloud.BasicDistributedZkTest.testDistribSearch

Error Message:
Captured an uncaught exception in thread: Thread[id=1513, name=testExecutor-610-thread-2, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1513, name=testExecutor-610-thread-2, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
Caused by: java.lang.RuntimeException: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:25218
	at __randomizedtesting.SeedInfo.seed([4E6781CA4B76983]:0)
	at org.apache.solr.cloud.BasicDistributedZkTest$2.run(BasicDistributedZkTest.java:706)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
	at java.lang.Thread.run(Thread.java:722)
Caused by: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:25218
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:412)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181)
	at org.apache.solr.cloud.BasicDistributedZkTest$2.run(BasicDistributedZkTest.java:704)
	... 3 more
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:150)
	at java.net.SocketInputStream.read(SocketInputStream.java:121)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:149)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:111)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:264)
	at org.apache.http.impl.conn.DefaultResponseParser.parseHead(DefaultResponseParser.java:98)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:252)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:282)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:247)
	at org.apache.http.impl.conn.AbstractClientConnAdapter.receiveResponseHeader(AbstractClientConnAdapter.java:216)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:298)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:647)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:464)
	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
	at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:732)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:353)
	... 5 more




Build Log:
[...truncated 9015 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
[junit4:junit4]   2> 0 T1041 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4]   2> 5 T1041 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZkTest-1359790608309
[junit4:junit4]   2> 6 T1041 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 7 T1042 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 10 T1042 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 10 T1042 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4]   2> 10 T1042 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4]   2> 11 T1042 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 12 T1042 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4]   2> 108 T1041 oasc.ZkTestServer.run start zk server on port:25193
[junit4:junit4]   2> 108 T1041 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:25193 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@389df1f6
[junit4:junit4]   2> 109 T1041 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 109 T1047 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:25193. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 110 T1047 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:25193, initiating session
[junit4:junit4]   2> 110 T1043 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:25194
[junit4:junit4]   2> 111 T1043 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:25194
[junit4:junit4]   2> 111 T1045 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 118 T1045 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c99d60fbc0000 with negotiated timeout 10000 for client /140.211.11.196:25194
[junit4:junit4]   2> 118 T1047 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:25193, sessionid = 0x13c99d60fbc0000, negotiated timeout = 10000
[junit4:junit4]   2> 118 T1048 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@389df1f6 name:ZooKeeperConnection Watcher:127.0.0.1:25193 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 118 T1041 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 119 T1041 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 122 T1046 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c99d60fbc0000
[junit4:junit4]   2> 123 T1048 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 123 T1041 oaz.ZooKeeper.close Session: 0x13c99d60fbc0000 closed
[junit4:junit4]   2> 123 T1043 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:25194 which had sessionid 0x13c99d60fbc0000
[junit4:junit4]   2> 123 T1041 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:25193/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@4773ff3a
[junit4:junit4]   2> 124 T1041 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 124 T1049 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:25193. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 125 T1049 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:25193, initiating session
[junit4:junit4]   2> 125 T1043 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:25195
[junit4:junit4]   2> 125 T1043 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:25195
[junit4:junit4]   2> 132 T1045 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c99d60fbc0001 with negotiated timeout 10000 for client /140.211.11.196:25195
[junit4:junit4]   2> 132 T1049 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:25193, sessionid = 0x13c99d60fbc0001, negotiated timeout = 10000
[junit4:junit4]   2> 132 T1050 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4773ff3a name:ZooKeeperConnection Watcher:127.0.0.1:25193/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 132 T1041 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 133 T1041 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 137 T1041 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 146 T1041 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 149 T1041 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 153 T1041 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 154 T1041 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 167 T1041 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 168 T1041 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 273 T1041 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 274 T1041 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 278 T1041 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 279 T1041 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 290 T1041 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 291 T1041 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 295 T1041 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 296 T1041 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 300 T1041 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 301 T1041 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 304 T1041 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 305 T1041 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 309 T1041 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 309 T1041 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 313 T1046 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c99d60fbc0001
[junit4:junit4]   2> 320 T1050 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 320 T1043 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:25195 which had sessionid 0x13c99d60fbc0001
[junit4:junit4]   2> 320 T1041 oaz.ZooKeeper.close Session: 0x13c99d60fbc0001 closed
[junit4:junit4]   2> 556 T1041 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 561 T1041 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:25198
[junit4:junit4]   2> 561 T1041 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 562 T1041 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 562 T1041 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359790608624
[junit4:junit4]   2> 563 T1041 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359790608624/solr.xml
[junit4:junit4]   2> 563 T1041 oasc.CoreContainer.<init> New CoreContainer 12500011
[junit4:junit4]   2> 564 T1041 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359790608624/'
[junit4:junit4]   2> 564 T1041 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359790608624/'
[junit4:junit4]   2> 619 T1041 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 620 T1041 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 620 T1041 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 621 T1041 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 621 T1041 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 622 T1041 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 622 T1041 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 623 T1041 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 623 T1041 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 624 T1041 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 646 T1041 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 665 T1041 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:25193/solr
[junit4:junit4]   2> 666 T1041 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 667 T1041 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:25193 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@34c6e1be
[junit4:junit4]   2> 668 T1041 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 668 T1060 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:25193. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 669 T1060 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:25193, initiating session
[junit4:junit4]   2> 669 T1043 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:25199
[junit4:junit4]   2> 670 T1043 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:25199
[junit4:junit4]   2> 672 T1045 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c99d60fbc0002 with negotiated timeout 20000 for client /140.211.11.196:25199
[junit4:junit4]   2> 672 T1060 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:25193, sessionid = 0x13c99d60fbc0002, negotiated timeout = 20000
[junit4:junit4]   2> 672 T1061 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@34c6e1be name:ZooKeeperConnection Watcher:127.0.0.1:25193 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 673 T1041 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 674 T1046 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c99d60fbc0002
[junit4:junit4]   2> 675 T1061 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 675 T1043 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:25199 which had sessionid 0x13c99d60fbc0002
[junit4:junit4]   2> 675 T1041 oaz.ZooKeeper.close Session: 0x13c99d60fbc0002 closed
[junit4:junit4]   2> 676 T1041 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 689 T1041 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:25193/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@78c3252b
[junit4:junit4]   2> 690 T1041 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 690 T1062 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:25193. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 691 T1062 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:25193, initiating session
[junit4:junit4]   2> 691 T1043 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:25200
[junit4:junit4]   2> 691 T1043 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:25200
[junit4:junit4]   2> 693 T1045 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c99d60fbc0003 with negotiated timeout 20000 for client /140.211.11.196:25200
[junit4:junit4]   2> 693 T1062 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:25193, sessionid = 0x13c99d60fbc0003, negotiated timeout = 20000
[junit4:junit4]   2> 693 T1063 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@78c3252b name:ZooKeeperConnection Watcher:127.0.0.1:25193/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 693 T1041 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 696 T1046 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c99d60fbc0003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 704 T1041 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 707 T1041 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:25198_
[junit4:junit4]   2> 708 T1046 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c99d60fbc0003 type:delete cxid:0x7 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:25198_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:25198_
[junit4:junit4]   2> 710 T1041 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:25198_
[junit4:junit4]   2> 719 T1041 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 732 T1046 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c99d60fbc0003 type:delete cxid:0x15 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 733 T1041 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 743 T1041 oasc.Overseer.start Overseer (id=89115237306597379-127.0.0.1:25198_-n_0000000000) starting
[junit4:junit4]   2> 744 T1046 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c99d60fbc0003 type:create cxid:0x1a zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 745 T1046 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c99d60fbc0003 type:create cxid:0x1b zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 747 T1046 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c99d60fbc0003 type:create cxid:0x1c zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 756 T1046 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c99d60fbc0003 type:create cxid:0x1d zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 758 T1065 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 759 T1041 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 769 T1041 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 772 T1064 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 777 T1066 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359790608624/collection1
[junit4:junit4]   2> 777 T1066 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 778 T1066 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 778 T1066 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 780 T1066 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359790608624/collection1/'
[junit4:junit4]   2> 781 T1066 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359790608624/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 782 T1066 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359790608624/collection1/lib/README' to classloader
[junit4:junit4]   2> 834 T1066 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 906 T1066 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1007 T1066 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 1013 T1066 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 1635 T1066 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1644 T1066 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1647 T1066 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1659 T1066 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1663 T1066 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1666 T1066 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1667 T1066 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1668 T1066 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359790608624/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/control/data/
[junit4:junit4]   2> 1668 T1066 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7dafbdd4
[junit4:junit4]   2> 1669 T1066 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 1670 T1066 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/control/data forceNew:false
[junit4:junit4]   2> 1670 T1066 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/control/data
[junit4:junit4]   2> 1670 T1066 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/control/data/index/
[junit4:junit4]   2> 1671 T1066 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1672 T1066 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/control/data/index forceNew:false
[junit4:junit4]   2> 1677 T1066 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@258ba273; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 1678 T1066 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 1678 T1066 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/control/data/index
[junit4:junit4]   2> 1678 T1066 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 1680 T1066 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1680 T1066 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 1680 T1066 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1681 T1066 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 1681 T1066 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1681 T1066 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1681 T1066 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1682 T1066 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1682 T1066 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1682 T1066 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 1683 T1066 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 1683 T1066 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1683 T1066 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 1683 T1066 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 1684 T1066 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 1684 T1066 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1684 T1066 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1685 T1066 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1685 T1066 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1685 T1066 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1685 T1066 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1686 T1066 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1686 T1066 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1686 T1066 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1687 T1066 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 1687 T1066 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 1695 T1066 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1698 T1066 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 1699 T1066 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 1701 T1066 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/control/data
[junit4:junit4]   2> 1702 T1066 oass.SolrIndexSearcher.<init> Opening Searcher@52bc7e5 main
[junit4:junit4]   2> 1703 T1066 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1703 T1066 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1704 T1066 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 1720 T1066 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 1777 T1067 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@52bc7e5 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1780 T1066 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 1780 T1066 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 1781 T1046 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c99d60fbc0003 type:create cxid:0x50 zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 2277 T1064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2277 T1064 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:25198_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:25198"}
[junit4:junit4]   2> 2278 T1064 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4]   2> 2281 T1064 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 2282 T1046 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c99d60fbc0003 type:create cxid:0x58 zxid:0x2c txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 2297 T1063 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 2788 T1066 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 2788 T1066 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:25198 collection:control_collection shard:shard1
[junit4:junit4]   2> 2789 T1066 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4]   2> 2806 T1046 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c99d60fbc0003 type:delete cxid:0x72 zxid:0x36 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4]   2> 2807 T1066 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 2807 T1046 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c99d60fbc0003 type:create cxid:0x73 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2809 T1066 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 2810 T1066 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 2810 T1066 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:25198/collection1/
[junit4:junit4]   2> 2811 T1066 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 2811 T1066 oasc.SyncStrategy.syncToMe http://127.0.0.1:25198/collection1/ has no replicas
[junit4:junit4]   2> 2811 T1066 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:25198/collection1/
[junit4:junit4]   2> 2811 T1066 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4]   2> 2816 T1046 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c99d60fbc0003 type:create cxid:0x7d zxid:0x3b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3802 T1064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3897 T1063 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 3941 T1066 oasc.ZkController.register We are http://127.0.0.1:25198/collection1/ and leader is http://127.0.0.1:25198/collection1/
[junit4:junit4]   2> 3941 T1066 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:25198
[junit4:junit4]   2> 3941 T1066 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 3941 T1066 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 3942 T1066 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 3945 T1066 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3946 T1041 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 3946 T1041 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 3947 T1041 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 3963 T1041 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 3978 T1041 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:25193/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@63b9af80
[junit4:junit4]   2> 3979 T1041 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 3979 T1068 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:25193. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 3980 T1068 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:25193, initiating session
[junit4:junit4]   2> 3980 T1043 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:25209
[junit4:junit4]   2> 3980 T1043 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:25209
[junit4:junit4]   2> 3982 T1045 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c99d60fbc0004 with negotiated timeout 10000 for client /140.211.11.196:25209
[junit4:junit4]   2> 3982 T1068 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:25193, sessionid = 0x13c99d60fbc0004, negotiated timeout = 10000
[junit4:junit4]   2> 3982 T1069 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@63b9af80 name:ZooKeeperConnection Watcher:127.0.0.1:25193/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 3982 T1041 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 3984 T1041 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 3988 T1041 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 4157 T1041 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 4160 T1041 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:25210
[junit4:junit4]   2> 4161 T1041 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 4161 T1041 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 4162 T1041 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359790612292
[junit4:junit4]   2> 4162 T1041 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359790612292/solr.xml
[junit4:junit4]   2> 4163 T1041 oasc.CoreContainer.<init> New CoreContainer 2131746288
[junit4:junit4]   2> 4163 T1041 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359790612292/'
[junit4:junit4]   2> 4164 T1041 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359790612292/'
[junit4:junit4]   2> 4219 T1041 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 4219 T1041 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 4220 T1041 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 4220 T1041 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 4221 T1041 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 4221 T1041 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 4222 T1041 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 4222 T1041 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 4222 T1041 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 4223 T1041 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 4244 T1041 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 4263 T1041 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:25193/solr
[junit4:junit4]   2> 4264 T1041 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 4264 T1041 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:25193 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@70a214c3
[junit4:junit4]   2> 4265 T1041 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4266 T1079 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:25193. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4266 T1079 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:25193, initiating session
[junit4:junit4]   2> 4266 T1043 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:25211
[junit4:junit4]   2> 4267 T1043 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:25211
[junit4:junit4]   2> 4273 T1045 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c99d60fbc0005 with negotiated timeout 20000 for client /140.211.11.196:25211
[junit4:junit4]   2> 4273 T1079 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:25193, sessionid = 0x13c99d60fbc0005, negotiated timeout = 20000
[junit4:junit4]   2> 4273 T1080 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@70a214c3 name:ZooKeeperConnection Watcher:127.0.0.1:25193 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4273 T1041 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4275 T1046 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c99d60fbc0005
[junit4:junit4]   2> 4284 T1080 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 4284 T1043 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:25211 which had sessionid 0x13c99d60fbc0005
[junit4:junit4]   2> 4284 T1041 oaz.ZooKeeper.close Session: 0x13c99d60fbc0005 closed
[junit4:junit4]   2> 4284 T1041 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 4295 T1041 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:25193/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@743a122c
[junit4:junit4]   2> 4296 T1041 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4297 T1081 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:25193. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4297 T1081 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:25193, initiating session
[junit4:junit4]   2> 4297 T1043 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:25212
[junit4:junit4]   2> 4298 T1043 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:25212
[junit4:junit4]   2> 4299 T1045 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c99d60fbc0006 with negotiated timeout 20000 for client /140.211.11.196:25212
[junit4:junit4]   2> 4299 T1081 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:25193, sessionid = 0x13c99d60fbc0006, negotiated timeout = 20000
[junit4:junit4]   2> 4299 T1082 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@743a122c name:ZooKeeperConnection Watcher:127.0.0.1:25193/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4299 T1041 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4301 T1046 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c99d60fbc0006 type:create cxid:0x1 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4302 T1046 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c99d60fbc0006 type:create cxid:0x2 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4310 T1041 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:25210_
[junit4:junit4]   2> 4311 T1046 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c99d60fbc0006 type:delete cxid:0x4 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:25210_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:25210_
[junit4:junit4]   2> 4312 T1041 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:25210_
[junit4:junit4]   2> 4315 T1063 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 4316 T1069 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 4317 T1063 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 4325 T1041 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 4332 T1083 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359790612292/collection1
[junit4:junit4]   2> 4332 T1083 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 4333 T1083 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 4333 T1083 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 4335 T1083 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359790612292/collection1/'
[junit4:junit4]   2> 4336 T1083 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359790612292/collection1/lib/README' to classloader
[junit4:junit4]   2> 4337 T1083 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359790612292/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 4389 T1083 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 4461 T1083 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 4562 T1083 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 4568 T1083 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 5182 T1083 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 5189 T1083 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 5192 T1083 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 5205 T1083 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5209 T1083 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5212 T1083 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5213 T1083 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5214 T1083 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359790612292/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty1/
[junit4:junit4]   2> 5214 T1083 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7dafbdd4
[junit4:junit4]   2> 5215 T1083 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 5216 T1083 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty1 forceNew:false
[junit4:junit4]   2> 5216 T1083 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty1
[junit4:junit4]   2> 5216 T1083 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty1/index/
[junit4:junit4]   2> 5217 T1083 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 5218 T1083 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty1/index forceNew:false
[junit4:junit4]   2> 5261 T1083 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7138a3eb; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 5261 T1083 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 5262 T1083 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty1/index
[junit4:junit4]   2> 5262 T1083 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 5264 T1083 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 5264 T1083 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 5264 T1083 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 5264 T1083 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 5265 T1083 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 5265 T1083 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 5265 T1083 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 5266 T1083 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 5266 T1083 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 5267 T1083 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 5267 T1083 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 5267 T1083 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 5268 T1083 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 5268 T1083 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 5268 T1083 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 5268 T1083 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 5269 T1083 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5269 T1083 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5270 T1083 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5270 T1083 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5270 T1083 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5271 T1083 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5271 T1083 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5271 T1083 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5272 T1083 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 5272 T1083 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 5281 T1083 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 5284 T1083 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 5285 T1083 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 5287 T1083 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty1
[junit4:junit4]   2> 5288 T1083 oass.SolrIndexSearcher.<init> Opening Searcher@24867ff5 main
[junit4:junit4]   2> 5288 T1083 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 5289 T1083 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 5289 T1083 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 5300 T1083 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 5348 T1084 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@24867ff5 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 5352 T1083 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 5352 T1083 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 5405 T1064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 5406 T1064 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:25198_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:25198"}
[junit4:junit4]   2> 5410 T1064 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:25210_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:25210"}
[junit4:junit4]   2> 5410 T1064 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4]   2> 5410 T1064 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 5423 T1082 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5423 T1069 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5423 T1063 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6360 T1083 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 6360 T1083 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:25210 collection:collection1 shard:shard1
[junit4:junit4]   2> 6361 T1083 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 6391 T1046 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c99d60fbc0006 type:delete cxid:0x43 zxid:0x5a txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 6392 T1083 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 6392 T1046 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c99d60fbc0006 type:create cxid:0x44 zxid:0x5b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6403 T1083 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 6403 T1083 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 6403 T1083 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:25210/collection1/
[junit4:junit4]   2> 6404 T1083 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 6404 T1083 oasc.SyncStrategy.syncToMe http://127.0.0.1:25210/collection1/ has no replicas
[junit4:junit4]   2> 6404 T1083 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:25210/collection1/
[junit4:junit4]   2> 6404 T1083 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 6409 T1046 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c99d60fbc0006 type:create cxid:0x4e zxid:0x5f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6936 T1064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6952 T1063 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6952 T1082 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6952 T1069 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6980 T1083 oasc.ZkController.register We are http://127.0.0.1:25210/collection1/ and leader is http://127.0.0.1:25210/collection1/
[junit4:junit4]   2> 6980 T1083 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:25210
[junit4:junit4]   2> 6980 T1083 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 6980 T1083 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 6981 T1083 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 6982 T1083 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6984 T1041 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 6984 T1041 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 6985 T1041 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 7169 T1041 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 7172 T1041 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:25215
[junit4:junit4]   2> 7173 T1041 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 7173 T1041 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 7174 T1041 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359790615303
[junit4:junit4]   2> 7174 T1041 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359790615303/solr.xml
[junit4:junit4]   2> 7175 T1041 oasc.CoreContainer.<init> New CoreContainer 2123172342
[junit4:junit4]   2> 7175 T1041 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359790615303/'
[junit4:junit4]   2> 7176 T1041 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359790615303/'
[junit4:junit4]   2> 7231 T1041 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 7231 T1041 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 7232 T1041 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 7232 T1041 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 7233 T1041 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 7233 T1041 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 7234 T1041 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 7234 T1041 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 7234 T1041 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 7235 T1041 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 7258 T1041 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 7277 T1041 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:25193/solr
[junit4:junit4]   2> 7277 T1041 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 7278 T1041 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:25193 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@6068556
[junit4:junit4]   2> 7279 T1041 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 7279 T1094 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:25193. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 7280 T1094 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:25193, initiating session
[junit4:junit4]   2> 7280 T1043 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:25216
[junit4:junit4]   2> 7281 T1043 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:25216
[junit4:junit4]   2> 7283 T1045 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c99d60fbc0007 with negotiated timeout 20000 for client /140.211.11.196:25216
[junit4:junit4]   2> 7283 T1094 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:25193, sessionid = 0x13c99d60fbc0007, negotiated timeout = 20000
[junit4:junit4]   2> 7283 T1095 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6068556 name:ZooKeeperConnection Watcher:127.0.0.1:25193 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 7284 T1041 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 7285 T1046 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c99d60fbc0007
[junit4:junit4]   2> 7297 T1095 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 7297 T1043 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:25216 which had sessionid 0x13c99d60fbc0007
[junit4:junit4]   2> 7297 T1041 oaz.ZooKeeper.close Session: 0x13c99d60fbc0007 closed
[junit4:junit4]   2> 7298 T1041 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 7311 T1041 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:25193/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@5d8387b0
[junit4:junit4]   2> 7312 T1041 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 7312 T1096 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:25193. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 7313 T1096 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:25193, initiating session
[junit4:junit4]   2> 7313 T1043 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:25217
[junit4:junit4]   2> 7313 T1043 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:25217
[junit4:junit4]   2> 7315 T1045 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c99d60fbc0008 with negotiated timeout 20000 for client /140.211.11.196:25217
[junit4:junit4]   2> 7315 T1096 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:25193, sessionid = 0x13c99d60fbc0008, negotiated timeout = 20000
[junit4:junit4]   2> 7315 T1097 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5d8387b0 name:ZooKeeperConnection Watcher:127.0.0.1:25193/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 7315 T1041 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 7316 T1046 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c99d60fbc0008 type:create cxid:0x1 zxid:0x6c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7318 T1046 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c99d60fbc0008 type:create cxid:0x2 zxid:0x6d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7326 T1041 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:25215_
[junit4:junit4]   2> 7327 T1046 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c99d60fbc0008 type:delete cxid:0x4 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:25215_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:25215_
[junit4:junit4]   2> 7328 T1041 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:25215_
[junit4:junit4]   2> 7330 T1063 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7330 T1082 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7331 T1069 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7331 T1069 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 7333 T1063 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7333 T1082 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7340 T1041 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 7348 T1098 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359790615303/collection1
[junit4:junit4]   2> 7348 T1098 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 7349 T1098 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 7349 T1098 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 7351 T1098 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359790615303/collection1/'
[junit4:junit4]   2> 7352 T1098 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359790615303/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 7353 T1098 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359790615303/collection1/lib/README' to classloader
[junit4:junit4]   2> 7409 T1098 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 7499 T1098 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 7600 T1098 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 7607 T1098 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 8237 T1098 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 8248 T1098 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 8253 T1098 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 8272 T1098 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8278 T1098 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8283 T1098 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8284 T1098 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8286 T1098 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359790615303/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty2/
[junit4:junit4]   2> 8286 T1098 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7dafbdd4
[junit4:junit4]   2> 8287 T1098 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 8288 T1098 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty2 forceNew:false
[junit4:junit4]   2> 8288 T1098 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty2
[junit4:junit4]   2> 8289 T1098 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty2/index/
[junit4:junit4]   2> 8289 T1098 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 8291 T1098 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty2/index forceNew:false
[junit4:junit4]   2> 8326 T1098 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@37ea0a9d; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 8327 T1098 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 8328 T1098 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty2/index
[junit4:junit4]   2> 8328 T1098 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 8331 T1098 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 8331 T1098 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 8331 T1098 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 8332 T1098 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 8332 T1098 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 8332 T1098 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 8333 T1098 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 8334 T1098 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 8334 T1098 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 8335 T1098 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 8335 T1098 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 8336 T1098 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 8336 T1098 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 8336 T1098 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 8337 T1098 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 8337 T1098 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 8338 T1098 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8339 T1098 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8339 T1098 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8340 T1098 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8340 T1098 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8341 T1098 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8341 T1098 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8342 T1098 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8342 T1098 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 8343 T1098 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 8358 T1098 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 8364 T1098 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 8366 T1098 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 8368 T1098 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty2
[junit4:junit4]   2> 8370 T1098 oass.SolrIndexSearcher.<init> Opening Searcher@3d20f045 main
[junit4:junit4]   2> 8371 T1098 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 8372 T1098 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 8372 T1098 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 8409 T1098 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 8464 T1064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8465 T1064 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:25210_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:25210"}
[junit4:junit4]   2> 8583 T1063 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8583 T1097 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8583 T1082 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8583 T1069 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8696 T1099 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3d20f045 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 8700 T1098 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 8700 T1098 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 10095 T1064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 10097 T1064 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:25215_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:25215"}
[junit4:junit4]   2> 10097 T1064 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 10097 T1064 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 10113 T1063 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10113 T1097 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10113 T1082 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10113 T1069 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10737 T1098 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 10737 T1098 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:25215 collection:collection1 shard:shard2
[junit4:junit4]   2> 10738 T1098 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4]   2> 10781 T1046 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c99d60fbc0008 type:delete cxid:0x44 zxid:0x7d txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
[junit4:junit4]   2> 10782 T1098 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 10783 T1046 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c99d60fbc0008 type:create cxid:0x45 zxid:0x7e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 10793 T1098 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 10793 T1098 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 10793 T1098 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:25215/collection1/
[junit4:junit4]   2> 10794 T1098 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 10794 T1098 oasc.SyncStrategy.syncToMe http://127.0.0.1:25215/collection1/ has no replicas
[junit4:junit4]   2> 10794 T1098 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:25215/collection1/
[junit4:junit4]   2> 10795 T1098 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4]   2> 10801 T1046 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c99d60fbc0008 type:create cxid:0x4e zxid:0x81 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 11623 T1064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11676 T1063 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11676 T1097 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11676 T1082 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11676 T1069 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11723 T1098 oasc.ZkController.register We are http://127.0.0.1:25215/collection1/ and leader is http://127.0.0.1:25215/collection1/
[junit4:junit4]   2> 11723 T1098 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:25215
[junit4:junit4]   2> 11724 T1098 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 11724 T1098 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 11724 T1098 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 11731 T1098 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11732 T1041 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 11733 T1041 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 11733 T1041 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 11923 T1041 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 11925 T1041 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:25218
[junit4:junit4]   2> 11927 T1041 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 11927 T1041 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 11928 T1041 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359790620052
[junit4:junit4]   2> 11929 T1041 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359790620052/solr.xml
[junit4:junit4]   2> 11929 T1041 oasc.CoreContainer.<init> New CoreContainer 541111535
[junit4:junit4]   2> 11930 T1041 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359790620052/'
[junit4:junit4]   2> 11930 T1041 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359790620052/'
[junit4:junit4]   2> 11996 T1041 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 11997 T1041 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 11997 T1041 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 11998 T1041 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 11998 T1041 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 11999 T1041 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 12000 T1041 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 12000 T1041 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 12001 T1041 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 12001 T1041 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 12026 T1041 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 12104 T1041 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:25193/solr
[junit4:junit4]   2> 12105 T1041 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 12106 T1041 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:25193 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@86fa21b
[junit4:junit4]   2> 12107 T1041 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 12107 T1109 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:25193. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 12109 T1109 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:25193, initiating session
[junit4:junit4]   2> 12109 T1043 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:25219
[junit4:junit4]   2> 12109 T1043 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:25219
[junit4:junit4]   2> 12112 T1045 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c99d60fbc0009 with negotiated timeout 20000 for client /140.211.11.196:25219
[junit4:junit4]   2> 12113 T1109 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:25193, sessionid = 0x13c99d60fbc0009, negotiated timeout = 20000
[junit4:junit4]   2> 12113 T1110 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@86fa21b name:ZooKeeperConnection Watcher:127.0.0.1:25193 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 12113 T1041 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 12116 T1046 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c99d60fbc0009
[junit4:junit4]   2> 12124 T1110 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 12124 T1043 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:25219 which had sessionid 0x13c99d60fbc0009
[junit4:junit4]   2> 12124 T1041 oaz.ZooKeeper.close Session: 0x13c99d60fbc0009 closed
[junit4:junit4]   2> 12125 T1041 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 12137 T1041 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:25193/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@37e0b7c5
[junit4:junit4]   2> 12138 T1041 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 12139 T1111 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:25193. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 12140 T1111 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:25193, initiating session
[junit4:junit4]   2> 12140 T1043 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:25220
[junit4:junit4]   2> 12140 T1043 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:25220
[junit4:junit4]   2> 12142 T1045 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c99d60fbc000a with negotiated timeout 20000 for client /140.211.11.196:25220
[junit4:junit4]   2> 12142 T1111 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:25193, sessionid = 0x13c99d60fbc000a, negotiated timeout = 20000
[junit4:junit4]   2> 12143 T1112 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@37e0b7c5 name:ZooKeeperConnection Watcher:127.0.0.1:25193/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 12143 T1041 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 12144 T1046 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c99d60fbc000a type:create cxid:0x1 zxid:0x8e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12146 T1046 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c99d60fbc000a type:create cxid:0x2 zxid:0x8f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12148 T1041 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:25218_
[junit4:junit4]   2> 12148 T1046 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c99d60fbc000a type:delete cxid:0x4 zxid:0x90 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:25218_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:25218_
[junit4:junit4]   2> 12156 T1041 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:25218_
[junit4:junit4]   2> 12159 T1063 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12159 T1097 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12159 T1082 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12160 T1069 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 12160 T1069 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12162 T1063 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 12162 T1082 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 12162 T1097 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 12167 T1041 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 12178 T1113 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359790620052/collection1
[junit4:junit4]   2> 12178 T1113 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 12180 T1113 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 12180 T1113 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 12182 T1113 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359790620052/collection1/'
[junit4:junit4]   2> 12184 T1113 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359790620052/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 12184 T1113 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359790620052/collection1/lib/README' to classloader
[junit4:junit4]   2> 12251 T1113 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 12333 T1113 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 12434 T1113 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 12441 T1113 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 13098 T1113 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 13110 T1113 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 13114 T1113 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 13135 T1113 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 13141 T1113 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 13146 T1113 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 13148 T1113 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 13150 T1113 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359790620052/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty3/
[junit4:junit4]   2> 13151 T1113 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7dafbdd4
[junit4:junit4]   2> 13151 T1113 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 13152 T1113 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty3 forceNew:false
[junit4:junit4]   2> 13153 T1113 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty3
[junit4:junit4]   2> 13153 T1113 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty3/index/
[junit4:junit4]   2> 13154 T1113 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 13155 T1113 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty3/index forceNew:false
[junit4:junit4]   2> 13190 T1064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 13191 T1064 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:25215_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:25215"}
[junit4:junit4]   2> 13206 T1113 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@17457476; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 13207 T1113 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 13208 T1113 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty3/index
[junit4:junit4]   2> 13208 T1113 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 13227 T1063 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13227 T1112 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13227 T1082 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13227 T1069 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13228 T1097 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13228 T1113 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 13229 T1113 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 13230 T1113 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 13230 T1113 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 13231 T1113 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 13231 T1113 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 13232 T1113 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 13232 T1113 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 13233 T1113 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 13233 T1113 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 13234 T1113 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 13235 T1113 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 13235 T1113 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 13235 T1113 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 13236 T1113 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 13236 T1113 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 13237 T1113 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13237 T1113 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13238 T1113 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13239 T1113 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13240 T1113 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13241 T1113 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13241 T1113 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13242 T1113 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13243 T1113 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 13243 T1113 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 13255 T1113 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 13260 T1113 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 13262 T1113 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 13264 T1113 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty3
[junit4:junit4]   2> 13266 T1113 oass.SolrIndexSearcher.<init> Opening Searcher@7a193673 main
[junit4:junit4]   2> 13267 T1113 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 13268 T1113 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 13268 T1113 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 13275 T1113 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 13336 T1114 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7a193673 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 13341 T1113 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 13342 T1113 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 14738 T1064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14739 T1064 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:25218_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:25218"}
[junit4:junit4]   2> 14739 T1064 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 14739 T1064 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 14758 T1063 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14758 T1097 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14758 T1082 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14758 T1112 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14758 T1069 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15346 T1113 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 15346 T1113 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:25218 collection:collection1 shard:shard1
[junit4:junit4]   2> 15381 T1113 oasc.ZkController.register We are http://127.0.0.1:25218/collection1/ and leader is http://127.0.0.1:25210/collection1/
[junit4:junit4]   2> 15381 T1113 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:25218
[junit4:junit4]   2> 15381 T1113 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 15382 T1113 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C64 name=collection1 org.apache.solr.core.SolrCore@6a98a358 url=http://127.0.0.1:25218/collection1 node=127.0.0.1:25218_ C64_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:25218_, base_url=http://127.0.0.1:25218}
[junit4:junit4]   2> 15389 T1115 C64 P25218 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 15390 T1113 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 15391 T1115 C64 P25218 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 15391 T1115 C64 P25218 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 15391 T1115 C64 P25218 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 15392 T1041 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 15393 T1041 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 15393 T1041 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 15393 T1115 C64 P25218 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 15622 T1041 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 15625 T1041 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:25222
[junit4:junit4]   2> 15626 T1041 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 15626 T1041 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 15627 T1041 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359790623721
[junit4:junit4]   2> 15628 T1041 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359790623721/solr.xml
[junit4:junit4]   2> 15628 T1041 oasc.CoreContainer.<init> New CoreContainer 1551312038
[junit4:junit4]   2> 15629 T1041 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359790623721/'
[junit4:junit4]   2> 15629 T1041 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359790623721/'
[junit4:junit4]   2> 15710 T1041 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 15711 T1041 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 15712 T1041 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 15712 T1041 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 15713 T1041 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 15713 T1041 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 15714 T1041 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 15714 T1041 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 15715 T1041 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 15715 T1041 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 15740 T1041 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 15760 T1041 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:25193/solr
[junit4:junit4]   2> 15761 T1041 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 15762 T1041 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:25193 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@7855ce96
[junit4:junit4]   2> 15763 T1041 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 15763 T1125 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:25193. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 15764 T1125 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:25193, initiating session
[junit4:junit4]   2> 15765 T1043 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:25223
[junit4:junit4]   2> 15765 T1043 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:25223
[junit4:junit4]   2> 15767 T1045 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c99d60fbc000b with negotiated timeout 20000 for client /140.211.11.196:25223
[junit4:junit4]   2> 15768 T1125 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:25193, sessionid = 0x13c99d60fbc000b, negotiated timeout = 20000
[junit4:junit4]   2> 15768 T1126 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7855ce96 name:ZooKeeperConnection Watcher:127.0.0.1:25193 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 15768 T1041 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 15770 T1046 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c99d60fbc000b
[junit4:junit4]   2> 15772 T1126 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 15772 T1043 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:25223 which had sessionid 0x13c99d60fbc000b
[junit4:junit4]   2> 15772 T1041 oaz.ZooKeeper.close Session: 0x13c99d60fbc000b closed
[junit4:junit4]   2> 15773 T1041 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 15785 T1041 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:25193/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@26ebb7fe
[junit4:junit4]   2> 15786 T1041 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 15787 T1127 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:25193. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 15787 T1127 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:25193, initiating session
[junit4:junit4]   2> 15788 T1043 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:25224
[junit4:junit4]   2> 15788 T1043 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:25224
[junit4:junit4]   2> 15790 T1045 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c99d60fbc000c with negotiated timeout 20000 for client /140.211.11.196:25224
[junit4:junit4]   2> 15790 T1127 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:25193, sessionid = 0x13c99d60fbc000c, negotiated timeout = 20000
[junit4:junit4]   2> 15790 T1128 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@26ebb7fe name:ZooKeeperConnection Watcher:127.0.0.1:25193/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 15791 T1041 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 15792 T1046 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c99d60fbc000c type:create cxid:0x1 zxid:0xa1 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 15794 T1046 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c99d60fbc000c type:create cxid:0x2 zxid:0xa2 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 15802 T1041 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:25222_
[junit4:junit4]   2> 15803 T1046 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c99d60fbc000c type:delete cxid:0x4 zxid:0xa3 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:25222_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:25222_
[junit4:junit4]   2> 15805 T1041 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:25222_
[junit4:junit4]   2> 15808 T1063 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15808 T1082 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15808 T1097 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15809 T1069 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 15809 T1112 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 15809 T1069 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15809 T1112 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15811 T1063 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 15811 T1082 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 15813 T1097 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 15815 T1041 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 15824 T1129 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359790623721/collection1
[junit4:junit4]   2> 15824 T1129 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 15826 T1129 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 15826 T1129 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 15828 T1129 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359790623721/collection1/'
[junit4:junit4]   2> 15829 T1129 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359790623721/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 15830 T1129 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359790623721/collection1/lib/README' to classloader
[junit4:junit4]   2> 15892 T1129 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 15976 T1129 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 16077 T1129 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 16085 T1129 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 16270 T1064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16272 T1064 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:25218_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:25218"}
[junit4:junit4]   2> 16287 T1063 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16287 T1128 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16287 T1082 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16287 T1112 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16287 T1097 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16287 T1069 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16788 T1129 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 16800 T1129 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 16804 T1129 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 16826 T1129 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 16832 T1129 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 16839 T1129 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 16843 T1129 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 16845 T1129 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359790623721/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty4/
[junit4:junit4]   2> 16845 T1129 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@7dafbdd4
[junit4:junit4]   2> 16846 T1129 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 16848 T1129 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty4 forceNew:false
[junit4:junit4]   2> 16848 T1129 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty4
[junit4:junit4]   2> 16849 T1129 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty4/index/
[junit4:junit4]   2> 16849 T1129 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 16851 T1129 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty4/index forceNew:false
[junit4:junit4]   2> 16886 T1129 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@54a4b295; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 16886 T1129 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 16888 T1129 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty4/index
[junit4:junit4]   2> 16888 T1129 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 16891 T1129 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 16891 T1129 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 16891 T1129 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 16892 T1129 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 16893 T1129 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 16893 T1129 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 16893 T1129 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 16894 T1129 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 16895 T1129 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 16896 T1129 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 16896 T1129 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 16897 T1129 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 16897 T1129 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 16898 T1129 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 16898 T1129 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 16899 T1129 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 16899 T1129 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16900 T1129 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16900 T1129 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16901 T1129 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16902 T1129 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16902 T1129 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16903 T1129 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16903 T1129 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16904 T1129 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 16905 T1129 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 16922 T1129 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 16927 T1129 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 16929 T1129 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 16932 T1129 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty4
[junit4:junit4]   2> 16935 T1129 oass.SolrIndexSearcher.<init> Opening Searcher@3a457baf main
[junit4:junit4]   2> 16936 T1129 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 16936 T1129 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 16937 T1129 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 16947 T1129 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 17077 T1130 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3a457baf main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 17082 T1129 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 17083 T1129 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 17797 T1064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17798 T1064 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:25222_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:25222"}
[junit4:junit4]   2> 17798 T1064 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 17798 T1064 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 17865 T1063 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17865 T1097 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17865 T1112 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17865 T1082 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17865 T1128 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17865 T1069 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18086 T1129 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 18086 T1129 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:25222 collection:collection1 shard:shard2
[junit4:junit4]   2> 18093 T1129 oasc.ZkController.register We are http://127.0.0.1:25222/collection1/ and leader is http://127.0.0.1:25215/collection1/
[junit4:junit4]   2> 18093 T1129 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:25222
[junit4:junit4]   2> 18094 T1129 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 18094 T1129 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C65 name=collection1 org.apache.solr.core.SolrCore@3cb9f61a url=http://127.0.0.1:25222/collection1 node=127.0.0.1:25222_ C65_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:25222_, base_url=http://127.0.0.1:25222}
[junit4:junit4]   2> 18094 T1131 C65 P25222 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 18095 T1131 C65 P25222 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 18095 T1129 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 18096 T1131 C65 P25222 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 18096 T1131 C65 P25222 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 18097 T1041 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 18097 T1041 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 18098 T1041 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 18098 T1131 C65 P25222 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 18121 T1041 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 18124 T1041 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4]   2> 18125 T1041 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C64_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:25218_, base_url=http://127.0.0.1:25218}
[junit4:junit4]   2> 18451 T1115 C64 P25218 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:25210/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 18463 T1115 C64 P25218 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 18483 T1115 C64 P25218 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:25218 START replicas=[http://127.0.0.1:25210/collection1/] nUpdates=100
[junit4:junit4]   2> 18484 T1115 C64 P25218 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 18485 T1115 C64 P25218 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 18485 T1115 C64 P25218 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 18485 T1115 C64 P25218 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 18485 T1115 C64 P25218 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 18485 T1115 C64 P25218 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:25210/collection1/. core=collection1
[junit4:junit4]   2> 18486 T1115 C64 P25218 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C66 name=collection1 org.apache.solr.core.SolrCore@73a8a36c url=http://127.0.0.1:25210/collection1 node=127.0.0.1:25210_ C66_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:25210_, base_url=http://127.0.0.1:25210, leader=true}
[junit4:junit4]   2> 18500 T1073 C66 P25210 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 18504 T1074 C66 P25210 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 18505 T1074 C66 P25210 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty1
[junit4:junit4]   2> 18511 T1074 C66 P25210 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7138a3eb; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 18512 T1074 C66 P25210 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 18517 T1074 C66 P25210 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7138a3eb; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7138a3eb; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 18518 T1074 C66 P25210 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 18519 T1074 C66 P25210 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty1
[junit4:junit4]   2> 18520 T1074 C66 P25210 oass.SolrIndexSearcher.<init> Opening Searcher@1ce8ad12 realtime
[junit4:junit4]   2> 18520 T1074 C66 P25210 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 18521 T1074 C66 P25210 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 18
[junit4:junit4]   2> 18590 T1115 C64 P25218 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 18590 T1115 C64 P25218 oascsi.HttpClientUtil.createClient Creating new http client, config:connTimeout=5000&socketTimeout=20000&allowCompression=false&maxConnections=10000&maxConnectionsPerHost=10000
[junit4:junit4]   2> 18621 T1115 C64 P25218 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 18624 T1075 C66 P25210 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 18625 T1075 C66 P25210 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 18626 T1115 C64 P25218 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 18627 T1115 C64 P25218 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 18627 T1115 C64 P25218 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 18630 T1075 C66 P25210 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty1
[junit4:junit4]   2> 18631 T1075 C66 P25210 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty1/index
[junit4:junit4]   2> 18631 T1075 C66 P25210 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=2 
[junit4:junit4]   2> 18632 T1115 C64 P25218 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 18634 T1115 C64 P25218 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty3/index.20130202033706936 forceNew:false
[junit4:junit4]   2> 18635 T1115 C64 P25218 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty3
[junit4:junit4]   2> 18637 T1115 C64 P25218 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty3/index.20130202033706936 lockFactory=org.apache.lucene.store.NativeFSLockFactory@17a13160; maxCacheMB=48.0 maxMergeSizeMB=4.0)) fullCopy=true
[junit4:junit4]   2> 18659 T1075 C66 P25210 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=8 
[junit4:junit4]   2> 18667 T1115 C64 P25218 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty3/index
[junit4:junit4]   2> 18667 T1115 C64 P25218 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 18684 T1115 C64 P25218 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130202033706936
[junit4:junit4]   2> 18687 T1115 C64 P25218 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty3
[junit4:junit4]   2> 18688 T1115 C64 P25218 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty3
[junit4:junit4]   2> 18688 T1115 C64 P25218 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty3/index/ new=./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty3/index.20130202033706936
[junit4:junit4]   2> 18692 T1115 C64 P25218 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty3/index.20130202033706936 lockFactory=org.apache.lucene.store.NativeFSLockFactory@17a13160; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 18692 T1115 C64 P25218 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 18694 T1115 C64 P25218 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty3
[junit4:junit4]   2> 18699 T1115 C64 P25218 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 18699 T1115 C64 P25218 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 18699 T1115 C64 P25218 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 18700 T1115 C64 P25218 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty3/index.20130202033706936
[junit4:junit4]   2> 18701 T1115 C64 P25218 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty3
[junit4:junit4]   2> 18705 T1115 C64 P25218 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty3/index.20130202033706936 lockFactory=org.apache.lucene.store.NativeFSLockFactory@17a13160; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 18705 T1115 C64 P25218 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 18705 T1115 C64 P25218 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 18706 T1115 C64 P25218 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty3
[junit4:junit4]   2> 18706 T1115 C64 P25218 oass.SolrIndexSearcher.<init> Opening Searcher@46259ab2 main
[junit4:junit4]   2> 18708 T1114 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@46259ab2 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 18709 T1114 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty3/index
[junit4:junit4]   2> 18709 T1115 C64 P25218 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 18743 T1115 C64 P25218 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty3/index.20130202033706936 lockFactory=org.apache.lucene.store.NativeFSLockFactory@17a13160; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty3/index.20130202033706936 lockFactory=org.apache.lucene.store.NativeFSLockFactory@17a13160; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 18743 T1115 C64 P25218 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 18745 T1115 C64 P25218 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty3
[junit4:junit4]   2> 18745 T1115 C64 P25218 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 18745 T1115 C64 P25218 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty3/index.20130202033706936
[junit4:junit4]   2> 18746 T1115 C64 P25218 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty3/index
[junit4:junit4]   2> 18746 T1115 C64 P25218 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 18746 T1115 C64 P25218 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 18746 T1115 C64 P25218 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 18746 T1115 C64 P25218 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 18749 T1115 C64 P25218 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 19128 T1041 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19373 T1064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19374 T1064 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:25222_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:25222"}
[junit4:junit4]   2> 19393 T1064 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:25218_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:25218"}
[junit4:junit4]   2> 19404 T1063 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19404 T1112 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19404 T1097 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19404 T1128 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19405 T1082 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19404 T1069 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20131 T1041 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 21134 T1041 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C65_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:25222_, base_url=http://127.0.0.1:25222}
[junit4:junit4]   2> 22125 T1131 C65 P25222 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:25215/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 22125 T1131 C65 P25222 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:25222 START replicas=[http://127.0.0.1:25215/collection1/] nUpdates=100
[junit4:junit4]   2> 22126 T1131 C65 P25222 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 22126 T1131 C65 P25222 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 22126 T1131 C65 P25222 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 22127 T1131 C65 P25222 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 22128 T1131 C65 P25222 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 22128 T1131 C65 P25222 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:25215/collection1/. core=collection1
[junit4:junit4]   2> 22129 T1131 C65 P25222 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 22137 T1041 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> ASYNC  NEW_CORE C67 name=collection1 org.apache.solr.core.SolrCore@33e77b29 url=http://127.0.0.1:25215/collection1 node=127.0.0.1:25215_ C67_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:25215_, base_url=http://127.0.0.1:25215, leader=true}
[junit4:junit4]   2> 22142 T1088 C67 P25215 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 22148 T1089 C67 P25215 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 22149 T1089 C67 P25215 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty2
[junit4:junit4]   2> 22153 T1089 C67 P25215 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@37ea0a9d; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 22154 T1089 C67 P25215 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 22159 T1089 C67 P25215 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@37ea0a9d; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@37ea0a9d; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 22160 T1089 C67 P25215 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 22161 T1089 C67 P25215 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty2
[junit4:junit4]   2> 22162 T1089 C67 P25215 oass.SolrIndexSearcher.<init> Opening Searcher@49af1488 realtime
[junit4:junit4]   2> 22162 T1089 C67 P25215 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 22162 T1089 C67 P25215 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 14
[junit4:junit4]   2> 22164 T1131 C65 P25222 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 22165 T1131 C65 P25222 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 22174 T1090 C67 P25215 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 22174 T1090 C67 P25215 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=5 
[junit4:junit4]   2> 22175 T1131 C65 P25222 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 22176 T1131 C65 P25222 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 22176 T1131 C65 P25222 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 22179 T1090 C67 P25215 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty2
[junit4:junit4]   2> 22180 T1090 C67 P25215 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty2/index
[junit4:junit4]   2> 22181 T1090 C67 P25215 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=3 
[junit4:junit4]   2> 22182 T1131 C65 P25222 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 22183 T1131 C65 P25222 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty4/index.20130202033710485 forceNew:false
[junit4:junit4]   2> 22184 T1131 C65 P25222 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty4
[junit4:junit4]   2> 22184 T1131 C65 P25222 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty4/index.20130202033710485 lockFactory=org.apache.lucene.store.NativeFSLockFactory@f481b2d; maxCacheMB=48.0 maxMergeSizeMB=4.0)) fullCopy=true
[junit4:junit4]   2> 22188 T1090 C67 P25215 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> 22190 T1131 C65 P25222 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty4/index
[junit4:junit4]   2> 22191 T1131 C65 P25222 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 22191 T1131 C65 P25222 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130202033710485
[junit4:junit4]   2> 22192 T1131 C65 P25222 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty4
[junit4:junit4]   2> 22193 T1131 C65 P25222 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty4
[junit4:junit4]   2> 22193 T1131 C65 P25222 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty4/index/ new=./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty4/index.20130202033710485
[junit4:junit4]   2> 22197 T1131 C65 P25222 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty4/index.20130202033710485 lockFactory=org.apache.lucene.store.NativeFSLockFactory@f481b2d; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 22198 T1131 C65 P25222 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 22199 T1131 C65 P25222 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty4
[junit4:junit4]   2> 22200 T1131 C65 P25222 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 22200 T1131 C65 P25222 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 22200 T1131 C65 P25222 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 22201 T1131 C65 P25222 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty4/index.20130202033710485
[junit4:junit4]   2> 22202 T1131 C65 P25222 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty4
[junit4:junit4]   2> 22206 T1131 C65 P25222 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty4/index.20130202033710485 lockFactory=org.apache.lucene.store.NativeFSLockFactory@f481b2d; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 22206 T1131 C65 P25222 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 22206 T1131 C65 P25222 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 22207 T1131 C65 P25222 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty4
[junit4:junit4]   2> 22207 T1131 C65 P25222 oass.SolrIndexSearcher.<init> Opening Searcher@7dcc35cc main
[junit4:junit4]   2> 22209 T1130 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7dcc35cc main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 22210 T1130 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty4/index
[junit4:junit4]   2> 22210 T1131 C65 P25222 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 22214 T1131 C65 P25222 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty4/index.20130202033710485 lockFactory=org.apache.lucene.store.NativeFSLockFactory@f481b2d; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty4/index.20130202033710485 lockFactory=org.apache.lucene.store.NativeFSLockFactory@f481b2d; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 22214 T1131 C65 P25222 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 22215 T1131 C65 P25222 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty4
[junit4:junit4]   2> 22215 T1131 C65 P25222 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 22216 T1131 C65 P25222 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty4/index.20130202033710485
[junit4:junit4]   2> 22216 T1131 C65 P25222 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty4/index
[junit4:junit4]   2> 22216 T1131 C65 P25222 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 22216 T1131 C65 P25222 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 22216 T1131 C65 P25222 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 22217 T1131 C65 P25222 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 22223 T1131 C65 P25222 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 22423 T1064 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 22425 T1064 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:25222_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:25222"}
[junit4:junit4]   2> 22443 T1063 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 22443 T1069 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 22443 T1082 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 22444 T1128 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 22443 T1097 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 22443 T1112 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 23141 T1041 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 23143 T1041 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4]   2> ASYNC  NEW_CORE C68 name=collection1 org.apache.solr.core.SolrCore@21cbd67d url=http://127.0.0.1:25198/collection1 node=127.0.0.1:25198_ C68_STATE=coll:control_collection core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:25198_, base_url=http://127.0.0.1:25198, leader=true}
[junit4:junit4]   2> 23157 T1053 C68 P25198 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/control/data
[junit4:junit4]   2> 23162 T1053 C68 P25198 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@258ba273; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 23162 T1053 C68 P25198 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 23164 T1053 C68 P25198 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/control/data
[junit4:junit4]   2> 23164 T1053 C68 P25198 oass.SolrIndexSearcher.<init> Opening Searcher@4cf772c9 realtime
[junit4:junit4]   2> 23165 T1053 C68 P25198 /update {wt=javabin&version=2} {deleteByQuery=*:* (-1425843821172752384)} 0 10
[junit4:junit4]   2> 23171 T1091 C67 P25215 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty2
[junit4:junit4]   2> 23178 T1076 C66 P25210 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty1
[junit4:junit4]   2>  C65_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:25222_, base_url=http://127.0.0.1:25222}
[junit4:junit4]   2> 23187 T1118 C65 P25222 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty4
[junit4:junit4]   2> 23188 T1118 C65 P25222 /update {update.distrib=FROMLEADER&_version_=-1425843821186383872&update.from=http://127.0.0.1:25215/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1425843821186383872)} 0 3
[junit4:junit4]   2>  C64_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:25218_, base_url=http://127.0.0.1:25218}
[junit4:junit4]   2> 23191 T1102 C64 P25218 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359790608308/jetty3
[junit4:junit4]   2> 23192 T1102 C64 P25218 /update {update.distrib=FROMLEADER&_version_=-1425843821193723904&update.from=http://127.0.0.1:25210/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1425843821193723904)} 0 3
[junit4:junit4]   2> 23193 T1076 C66 P25210 /update {update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1425843821193723904)} 0 17
[junit4:junit4]   2> 23194 T1091 C67 P25215 /update {wt=javabin&version=2} {deleteByQuery=*:* (-1425843821186383872)} 0 25
[junit4:junit4]   2> 23203 T1053 C68 P25198 /update {wt=javabin&version=2} {add=[1 (1425843821217841152)]} 0 5
[junit4:junit4]   2> 23216 T1102 C64 P25218 /update {distrib.from=http://127.0.0.1:25210/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1425843821226229760)]} 0 3
[junit4:junit4]   2> 23217 T1077 C66 P25210 /update {wt=javabin&version=2} {add=[1 (1425843821226229760)]} 0 11
[junit4:junit4]   2> 23223 T1053 C68 P25198 /update {wt=javabin&version=2} {add=[2 (1425843821241958400)]} 0 2
[junit4:junit4]   2> 23245 T1118 C65 P25222 /update {distrib.from=http://127.0.0.1:25215/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1425843821256638464)]} 0 4
[junit4:junit4]   2> 23252 T1092 C67 P25215 /update {distrib.from=http://127.0.0.1:25210/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[2 (1425843821256638464)]} 0 17
[junit4:junit4]   2> 23253 T1077 C66 P25210 /update {wt=javabin&version=2} {add

[...truncated too long message...]

change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 482343 T1128 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> 482343 T1128 oascc.ZkStateReader$3.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> 482344 T1128 oasc.LeaderElector$1.process WARNING  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection2/leader_elect/shard2/election
[junit4:junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:236)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:233)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:233)
[junit4:junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:84)
[junit4:junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4:junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4]   2> 
[junit4:junit4]   2> 482344 T1128 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 482344 T1128 oascc.ZkStateReader$2.process WARNING ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4]   2> 482345 T1128 oasc.LeaderElector$1.process WARNING  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
[junit4:junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:236)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:233)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:233)
[junit4:junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:84)
[junit4:junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4:junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4]   2> 
[junit4:junit4]   2> 482345 T1128 oasc.LeaderElector$1.process WARNING  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/unloadcollection/leader_elect/shard1/election
[junit4:junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:236)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:233)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:233)
[junit4:junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:84)
[junit4:junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4:junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4]   2> 
[junit4:junit4]   2> 482346 T1128 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> NOTE: test params are: codec=Lucene42: {n_f1=PostingsFormat(name=Memory doPackFST= true), foo_b=PostingsFormat(name=SimpleText), cat=PostingsFormat(name=Memory doPackFST= false), foo_d=MockVariableIntBlock(baseBlockSize=18), foo_f=PostingsFormat(name=SimpleText), n_tl1=PostingsFormat(name=Memory doPackFST= true), n_d1=PostingsFormat(name=Memory doPackFST= false), rnd_b=MockVariableIntBlock(baseBlockSize=18), intDefault=PostingsFormat(name=Memory doPackFST= false), n_td1=PostingsFormat(name=Memory doPackFST= true), timestamp=PostingsFormat(name=Memory doPackFST= false), id=MockVariableIntBlock(baseBlockSize=18), range_facet_sl=PostingsFormat(name=SimpleText), range_facet_si=PostingsFormat(name=Memory doPackFST= true), oddField_s=MockVariableIntBlock(baseBlockSize=18), sequence_i=MockVariableIntBlock(baseBlockSize=18), name=MockVariableIntBlock(baseBlockSize=18), foo_i=PostingsFormat(name=Memory doPackFST= true), regex_dup_B_s=PostingsFormat(name=Memory doPackFST= false), multiDefault=PostingsFormat(name=Memory doPackFST= true), n_tf1=PostingsFormat(name=Memory doPackFST= false), n_dt1=PostingsFormat(name=Memory doPackFST= true), genre_s=MockVariableIntBlock(baseBlockSize=18), author_t=PostingsFormat(name=Memory doPackFST= true), n_ti1=MockVariableIntBlock(baseBlockSize=18), range_facet_l=PostingsFormat(name=Memory doPackFST= false), text=PostingsFormat(name=SimpleText), _version_=PostingsFormat(name=Memory doPackFST= false), val_i=MockVariableIntBlock(baseBlockSize=18), SubjectTerms_mfacet=PostingsFormat(name=Memory doPackFST= false), series_t=PostingsFormat(name=Memory doPackFST= true), a_t=PostingsFormat(name=Memory doPackFST= false), n_tdt1=PostingsFormat(name=Memory doPackFST= false), regex_dup_A_s=MockVariableIntBlock(baseBlockSize=18), price=PostingsFormat(name=SimpleText), other_tl1=PostingsFormat(name=Memory doPackFST= false), n_l1=PostingsFormat(name=Memory doPackFST= false), a_si=PostingsFormat(name=Memory doPackFST= true), inStock=MockVariableIntBlock(baseBlockSize=18)}, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=ar_YE, timezone=America/Virgin
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=3,free=88891824,total=422576128
[junit4:junit4]   2> NOTE: All tests run in this JVM: [TestJoin, SpellCheckCollatorTest, TestFaceting, TestStressVersions, SpellPossibilityIteratorTest, DistributedSpellCheckComponentTest, DefaultValueUpdateProcessorTest, NoCacheHeaderTest, MoreLikeThisHandlerTest, CacheHeaderTest, TestLMDirichletSimilarityFactory, TestIndexSearcher, JSONWriterTest, TestPostingsSolrHighlighter, TestPHPSerializedResponseWriter, AnalysisAfterCoreReloadTest, TestGroupingSearch, DistributedTermsComponentTest, MBeansHandlerTest, TestQueryTypes, QueryElevationComponentTest, DateFieldTest, SimplePostToolTest, TestCharFilters, CurrencyFieldTest, WordBreakSolrSpellCheckerTest, DistanceFunctionTest, SortByFunctionTest, TestAtomicUpdateErrorCases, TestSuggestSpellingConverter, ChaosMonkeySafeLeaderTest, TestCollationField, TestLuceneMatchVersion, TestJmxMonitoredMap, TestConfig, TestComponentsName, ClusterStateUpdateTest, SimpleFacetsTest, SolrPluginUtilsTest, SolrRequestParserTest, TestDocumentBuilder, ZkControllerTest, SuggesterFSTTest, PreAnalyzedFieldTest, TestPerFieldSimilarity, TestLMJelinekMercerSimilarityFactory, DirectUpdateHandlerOptimizeTest, UpdateRequestProcessorFactoryTest, TestFastLRUCache, TestDefaultSimilarityFactory, TestWordDelimiterFilterFactory, RequestHandlersTest, TestPseudoReturnFields, TestFastOutputStream, SolrCmdDistributorTest, ScriptEngineTest, TestRandomFaceting, IndexBasedSpellCheckerTest, BasicDistributedZkTest]
[junit4:junit4] Completed on J0 in 482.99s, 1 test, 1 error <<< FAILURES!

[...truncated 385 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:346: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/common-build.xml:428: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:1176: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:841: There were test failures: 249 suites, 1038 tests, 1 error, 23 ignored (5 assumptions)

Total time: 42 minutes 43 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-trunk-java7 - Build # 3700 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-java7/3700/

2 tests failed.
REGRESSION:  org.apache.solr.cloud.BasicDistributedZkTest.testDistribSearch

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([5CD2923B7FED216C]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([5CD2923B7FED216C]:0)




Build Log:
[...truncated 9426 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
[junit4:junit4]   2> 0 T771 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4]   2> 6 T771 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZkTest-1359765085881
[junit4:junit4]   2> 7 T771 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 7 T772 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 10 T772 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 10 T772 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4]   2> 11 T772 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4]   2> 11 T772 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 12 T772 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4]   2> 108 T771 oasc.ZkTestServer.run start zk server on port:35058
[junit4:junit4]   2> 108 T771 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:35058 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@2ca061d7
[junit4:junit4]   2> 109 T771 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 109 T777 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:35058. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 110 T777 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:35058, initiating session
[junit4:junit4]   2> 110 T773 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:46529
[junit4:junit4]   2> 111 T773 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:46529
[junit4:junit4]   2> 111 T775 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 114 T775 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c98509ebf0000 with negotiated timeout 10000 for client /140.211.11.196:46529
[junit4:junit4]   2> 114 T777 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:35058, sessionid = 0x13c98509ebf0000, negotiated timeout = 10000
[junit4:junit4]   2> 115 T778 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2ca061d7 name:ZooKeeperConnection Watcher:127.0.0.1:35058 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 115 T771 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 115 T771 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 118 T776 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c98509ebf0000
[junit4:junit4]   2> 130 T778 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 130 T771 oaz.ZooKeeper.close Session: 0x13c98509ebf0000 closed
[junit4:junit4]   2> 131 T773 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:46529 which had sessionid 0x13c98509ebf0000
[junit4:junit4]   2> 131 T771 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:35058/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@4193907c
[junit4:junit4]   2> 132 T771 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 132 T779 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:35058. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 133 T779 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:35058, initiating session
[junit4:junit4]   2> 133 T773 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:37759
[junit4:junit4]   2> 133 T773 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:37759
[junit4:junit4]   2> 135 T775 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c98509ebf0001 with negotiated timeout 10000 for client /140.211.11.196:37759
[junit4:junit4]   2> 135 T779 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:35058, sessionid = 0x13c98509ebf0001, negotiated timeout = 10000
[junit4:junit4]   2> 135 T780 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4193907c name:ZooKeeperConnection Watcher:127.0.0.1:35058/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 135 T771 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 135 T771 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 139 T771 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 149 T771 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 152 T771 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 161 T771 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 162 T771 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 176 T771 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 176 T771 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 282 T771 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 283 T771 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 287 T771 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 287 T771 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 291 T771 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 292 T771 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 296 T771 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 296 T771 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 300 T771 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 301 T771 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 306 T771 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 306 T771 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 310 T771 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 311 T771 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 315 T776 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c98509ebf0001
[junit4:junit4]   2> 316 T780 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 316 T773 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:37759 which had sessionid 0x13c98509ebf0001
[junit4:junit4]   2> 316 T771 oaz.ZooKeeper.close Session: 0x13c98509ebf0001 closed
[junit4:junit4]   2> 462 T771 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 466 T771 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:65305
[junit4:junit4]   2> 466 T771 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 467 T771 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 467 T771 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359765086192
[junit4:junit4]   2> 468 T771 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359765086192/solr.xml
[junit4:junit4]   2> 468 T771 oasc.CoreContainer.<init> New CoreContainer 1045629780
[junit4:junit4]   2> 469 T771 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359765086192/'
[junit4:junit4]   2> 469 T771 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359765086192/'
[junit4:junit4]   2> 531 T771 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 532 T771 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 532 T771 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 533 T771 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 533 T771 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 534 T771 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 534 T771 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 534 T771 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 535 T771 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 535 T771 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 559 T771 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 579 T771 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:35058/solr
[junit4:junit4]   2> 580 T771 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 581 T771 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:35058 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@608ee7c0
[junit4:junit4]   2> 582 T771 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 582 T790 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:35058. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 583 T790 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:35058, initiating session
[junit4:junit4]   2> 583 T773 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:22080
[junit4:junit4]   2> 584 T773 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:22080
[junit4:junit4]   2> 586 T775 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c98509ebf0002 with negotiated timeout 20000 for client /140.211.11.196:22080
[junit4:junit4]   2> 586 T790 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:35058, sessionid = 0x13c98509ebf0002, negotiated timeout = 20000
[junit4:junit4]   2> 586 T791 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@608ee7c0 name:ZooKeeperConnection Watcher:127.0.0.1:35058 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 586 T771 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 588 T776 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c98509ebf0002
[junit4:junit4]   2> 589 T791 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 589 T773 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:22080 which had sessionid 0x13c98509ebf0002
[junit4:junit4]   2> 589 T771 oaz.ZooKeeper.close Session: 0x13c98509ebf0002 closed
[junit4:junit4]   2> 590 T771 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 603 T771 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:35058/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@3408e9a
[junit4:junit4]   2> 604 T771 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 605 T792 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:35058. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 605 T792 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:35058, initiating session
[junit4:junit4]   2> 605 T773 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:56893
[junit4:junit4]   2> 606 T773 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:56893
[junit4:junit4]   2> 607 T775 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c98509ebf0003 with negotiated timeout 20000 for client /140.211.11.196:56893
[junit4:junit4]   2> 607 T792 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:35058, sessionid = 0x13c98509ebf0003, negotiated timeout = 20000
[junit4:junit4]   2> 607 T793 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3408e9a name:ZooKeeperConnection Watcher:127.0.0.1:35058/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 608 T771 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 610 T776 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c98509ebf0003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 618 T771 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 622 T771 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:65305_
[junit4:junit4]   2> 623 T776 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c98509ebf0003 type:delete cxid:0x7 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:65305_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:65305_
[junit4:junit4]   2> 624 T771 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:65305_
[junit4:junit4]   2> 633 T771 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 646 T776 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c98509ebf0003 type:delete cxid:0x15 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 647 T771 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 651 T771 oasc.Overseer.start Overseer (id=89113564668690435-127.0.0.1:65305_-n_0000000000) starting
[junit4:junit4]   2> 652 T776 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c98509ebf0003 type:create cxid:0x1a zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 654 T776 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c98509ebf0003 type:create cxid:0x1b zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 662 T776 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c98509ebf0003 type:create cxid:0x1c zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 664 T776 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c98509ebf0003 type:create cxid:0x1d zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 665 T795 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 667 T771 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 678 T771 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 681 T794 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 686 T796 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359765086192/collection1
[junit4:junit4]   2> 686 T796 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 687 T796 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 687 T796 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 689 T796 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359765086192/collection1/'
[junit4:junit4]   2> 691 T796 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359765086192/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 691 T796 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359765086192/collection1/lib/README' to classloader
[junit4:junit4]   2> 750 T796 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 825 T796 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 926 T796 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 934 T796 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 1652 T796 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1664 T796 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1669 T796 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1690 T796 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1695 T796 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1700 T796 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1702 T796 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1703 T796 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359765086192/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/control/data/
[junit4:junit4]   2> 1704 T796 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@75fee96
[junit4:junit4]   2> 1704 T796 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 1742 T796 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/control/data forceNew:false
[junit4:junit4]   2> 1742 T796 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/control/data
[junit4:junit4]   2> 1743 T796 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/control/data/index/
[junit4:junit4]   2> 1743 T796 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1744 T796 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/control/data/index forceNew:false
[junit4:junit4]   2> 1776 T796 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@20ffca31; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 1777 T796 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 1778 T796 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/control/data/index
[junit4:junit4]   2> 1778 T796 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 1780 T796 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1780 T796 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 1781 T796 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1781 T796 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 1782 T796 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1782 T796 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1782 T796 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1783 T796 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1783 T796 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1784 T796 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 1784 T796 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 1785 T796 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1785 T796 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 1785 T796 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 1785 T796 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 1786 T796 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1786 T796 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1787 T796 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1787 T796 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1788 T796 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1788 T796 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1789 T796 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1789 T796 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1789 T796 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1790 T796 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 1790 T796 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 1803 T796 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1808 T796 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 1809 T796 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 1812 T796 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/control/data
[junit4:junit4]   2> 1814 T796 oass.SolrIndexSearcher.<init> Opening Searcher@658a19e5 main
[junit4:junit4]   2> 1815 T796 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1815 T796 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1815 T796 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 1833 T796 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 1891 T797 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@658a19e5 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1896 T796 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 1896 T796 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 1897 T776 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c98509ebf0003 type:create cxid:0x50 zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 2186 T794 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2187 T794 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:65305_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:65305"}
[junit4:junit4]   2> 2187 T794 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4]   2> 2193 T794 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 2193 T776 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c98509ebf0003 type:create cxid:0x58 zxid:0x2c txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 2211 T793 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 2909 T796 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 2909 T796 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:65305 collection:control_collection shard:shard1
[junit4:junit4]   2> 2910 T796 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4]   2> 2939 T776 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c98509ebf0003 type:delete cxid:0x72 zxid:0x36 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4]   2> 2940 T796 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 2941 T776 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c98509ebf0003 type:create cxid:0x73 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2943 T796 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 2943 T796 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 2944 T796 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:65305/collection1/
[junit4:junit4]   2> 2944 T796 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 2944 T796 oasc.SyncStrategy.syncToMe http://127.0.0.1:65305/collection1/ has no replicas
[junit4:junit4]   2> 2945 T796 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:65305/collection1/
[junit4:junit4]   2> 2945 T796 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4]   2> 2950 T776 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c98509ebf0003 type:create cxid:0x7d zxid:0x3b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3722 T794 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3769 T793 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 3819 T796 oasc.ZkController.register We are http://127.0.0.1:65305/collection1/ and leader is http://127.0.0.1:65305/collection1/
[junit4:junit4]   2> 3819 T796 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:65305
[junit4:junit4]   2> 3819 T796 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 3819 T796 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 3819 T796 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 3822 T796 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3823 T771 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 3824 T771 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 3825 T771 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 3844 T771 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 3862 T771 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:35058/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@360ee6a1
[junit4:junit4]   2> 3863 T771 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 3864 T798 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:35058. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 3864 T798 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:35058, initiating session
[junit4:junit4]   2> 3865 T773 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:20606
[junit4:junit4]   2> 3865 T773 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:20606
[junit4:junit4]   2> 3866 T775 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c98509ebf0004 with negotiated timeout 10000 for client /140.211.11.196:20606
[junit4:junit4]   2> 3866 T798 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:35058, sessionid = 0x13c98509ebf0004, negotiated timeout = 10000
[junit4:junit4]   2> 3867 T799 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@360ee6a1 name:ZooKeeperConnection Watcher:127.0.0.1:35058/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 3867 T771 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 3868 T771 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 3873 T771 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 4069 T771 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 4072 T771 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:35236
[junit4:junit4]   2> 4073 T771 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 4074 T771 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 4074 T771 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359765089748
[junit4:junit4]   2> 4075 T771 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359765089748/solr.xml
[junit4:junit4]   2> 4075 T771 oasc.CoreContainer.<init> New CoreContainer 892721855
[junit4:junit4]   2> 4076 T771 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359765089748/'
[junit4:junit4]   2> 4076 T771 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359765089748/'
[junit4:junit4]   2> 4145 T771 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 4145 T771 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 4146 T771 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 4146 T771 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 4147 T771 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 4147 T771 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 4148 T771 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 4148 T771 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 4149 T771 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 4149 T771 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 4174 T771 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 4195 T771 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:35058/solr
[junit4:junit4]   2> 4195 T771 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 4196 T771 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:35058 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@98ae2c2
[junit4:junit4]   2> 4197 T771 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4197 T809 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:35058. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4198 T809 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:35058, initiating session
[junit4:junit4]   2> 4199 T773 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:26197
[junit4:junit4]   2> 4199 T773 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:26197
[junit4:junit4]   2> 4201 T775 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c98509ebf0005 with negotiated timeout 20000 for client /140.211.11.196:26197
[junit4:junit4]   2> 4201 T809 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:35058, sessionid = 0x13c98509ebf0005, negotiated timeout = 20000
[junit4:junit4]   2> 4202 T810 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@98ae2c2 name:ZooKeeperConnection Watcher:127.0.0.1:35058 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4202 T771 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4204 T776 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c98509ebf0005
[junit4:junit4]   2> 4210 T810 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 4210 T773 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:26197 which had sessionid 0x13c98509ebf0005
[junit4:junit4]   2> 4210 T771 oaz.ZooKeeper.close Session: 0x13c98509ebf0005 closed
[junit4:junit4]   2> 4211 T771 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 4225 T771 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:35058/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@5ca4910d
[junit4:junit4]   2> 4226 T771 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4226 T811 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:35058. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4227 T811 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:35058, initiating session
[junit4:junit4]   2> 4227 T773 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:42963
[junit4:junit4]   2> 4228 T773 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:42963
[junit4:junit4]   2> 4230 T775 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c98509ebf0006 with negotiated timeout 20000 for client /140.211.11.196:42963
[junit4:junit4]   2> 4230 T811 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:35058, sessionid = 0x13c98509ebf0006, negotiated timeout = 20000
[junit4:junit4]   2> 4230 T812 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5ca4910d name:ZooKeeperConnection Watcher:127.0.0.1:35058/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4230 T771 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4231 T776 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c98509ebf0006 type:create cxid:0x1 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4233 T776 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c98509ebf0006 type:create cxid:0x2 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4241 T771 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35236_
[junit4:junit4]   2> 4242 T776 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c98509ebf0006 type:delete cxid:0x4 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:35236_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:35236_
[junit4:junit4]   2> 4243 T771 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:35236_
[junit4:junit4]   2> 4247 T793 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 4247 T799 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 4247 T793 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 4252 T771 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 4260 T813 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359765089748/collection1
[junit4:junit4]   2> 4260 T813 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 4261 T813 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 4261 T813 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 4263 T813 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359765089748/collection1/'
[junit4:junit4]   2> 4264 T813 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359765089748/collection1/lib/README' to classloader
[junit4:junit4]   2> 4265 T813 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359765089748/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 4322 T813 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 4403 T813 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 4504 T813 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 4512 T813 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 5198 T813 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 5209 T813 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 5214 T813 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 5232 T813 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5238 T813 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5244 T813 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5245 T813 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5247 T813 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359765089748/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty1/
[junit4:junit4]   2> 5248 T813 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@75fee96
[junit4:junit4]   2> 5248 T813 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 5249 T813 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty1 forceNew:false
[junit4:junit4]   2> 5250 T813 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty1
[junit4:junit4]   2> 5250 T813 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty1/index/
[junit4:junit4]   2> 5251 T813 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 5252 T813 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty1/index forceNew:false
[junit4:junit4]   2> 5260 T813 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@cdcf3d9; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 5260 T813 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 5261 T813 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty1/index
[junit4:junit4]   2> 5262 T813 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 5264 T813 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 5264 T813 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 5265 T813 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 5265 T813 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 5266 T813 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 5266 T813 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 5266 T813 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 5267 T813 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 5268 T813 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 5268 T813 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 5269 T813 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 5269 T813 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 5270 T813 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 5270 T813 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 5270 T813 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 5271 T813 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 5272 T813 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5272 T813 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5273 T813 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5273 T813 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5274 T813 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5274 T813 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5275 T813 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5276 T813 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5276 T813 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 5277 T813 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 5283 T794 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 5284 T794 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:65305_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:65305"}
[junit4:junit4]   2> 5291 T813 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 5296 T813 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 5297 T793 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5298 T799 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5297 T812 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5299 T813 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 5302 T813 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty1
[junit4:junit4]   2> 5304 T813 oass.SolrIndexSearcher.<init> Opening Searcher@2a7c878a main
[junit4:junit4]   2> 5305 T813 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 5305 T813 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 5306 T813 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 5314 T813 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 5368 T814 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2a7c878a main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 5374 T813 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 5374 T813 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 6804 T794 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6805 T794 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:35236_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:35236"}
[junit4:junit4]   2> 6805 T794 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4]   2> 6805 T794 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 6824 T793 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6824 T812 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6824 T799 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7378 T813 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 7378 T813 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:35236 collection:collection1 shard:shard1
[junit4:junit4]   2> 7379 T813 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 7399 T776 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c98509ebf0006 type:delete cxid:0x45 zxid:0x5b txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 7400 T813 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 7401 T776 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c98509ebf0006 type:create cxid:0x46 zxid:0x5c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7414 T813 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 7414 T813 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 7414 T813 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:35236/collection1/
[junit4:junit4]   2> 7414 T813 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 7415 T813 oasc.SyncStrategy.syncToMe http://127.0.0.1:35236/collection1/ has no replicas
[junit4:junit4]   2> 7415 T813 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:35236/collection1/
[junit4:junit4]   2> 7415 T813 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 7422 T776 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c98509ebf0006 type:create cxid:0x50 zxid:0x60 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8329 T794 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8541 T793 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8541 T799 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8542 T812 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8547 T813 oasc.ZkController.register We are http://127.0.0.1:35236/collection1/ and leader is http://127.0.0.1:35236/collection1/
[junit4:junit4]   2> 8547 T813 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:35236
[junit4:junit4]   2> 8547 T813 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 8547 T813 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 8548 T813 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 8554 T813 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8555 T771 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 8555 T771 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 8556 T771 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 8765 T771 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 8768 T771 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:38700
[junit4:junit4]   2> 8769 T771 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 8769 T771 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 8770 T771 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359765094449
[junit4:junit4]   2> 8770 T771 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359765094449/solr.xml
[junit4:junit4]   2> 8771 T771 oasc.CoreContainer.<init> New CoreContainer 989901818
[junit4:junit4]   2> 8771 T771 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359765094449/'
[junit4:junit4]   2> 8772 T771 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359765094449/'
[junit4:junit4]   2> 8835 T771 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 8835 T771 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 8836 T771 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 8836 T771 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 8837 T771 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 8837 T771 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 8838 T771 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 8838 T771 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 8839 T771 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 8839 T771 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 8862 T771 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 8881 T771 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:35058/solr
[junit4:junit4]   2> 8882 T771 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 8883 T771 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:35058 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@434ab4fa
[junit4:junit4]   2> 8884 T771 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 8884 T824 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:35058. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 8885 T824 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:35058, initiating session
[junit4:junit4]   2> 8885 T773 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:38702
[junit4:junit4]   2> 8886 T773 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:38702
[junit4:junit4]   2> 8888 T775 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c98509ebf0007 with negotiated timeout 20000 for client /140.211.11.196:38702
[junit4:junit4]   2> 8888 T824 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:35058, sessionid = 0x13c98509ebf0007, negotiated timeout = 20000
[junit4:junit4]   2> 8888 T825 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@434ab4fa name:ZooKeeperConnection Watcher:127.0.0.1:35058 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 8888 T771 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 8890 T776 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c98509ebf0007
[junit4:junit4]   2> 8891 T825 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 8891 T773 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:38702 which had sessionid 0x13c98509ebf0007
[junit4:junit4]   2> 8891 T771 oaz.ZooKeeper.close Session: 0x13c98509ebf0007 closed
[junit4:junit4]   2> 8892 T771 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 8905 T771 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:35058/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@48aa8a97
[junit4:junit4]   2> 8906 T771 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 8907 T826 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:35058. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 8907 T826 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:35058, initiating session
[junit4:junit4]   2> 8908 T773 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:38703
[junit4:junit4]   2> 8908 T773 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:38703
[junit4:junit4]   2> 8910 T775 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c98509ebf0008 with negotiated timeout 20000 for client /140.211.11.196:38703
[junit4:junit4]   2> 8910 T826 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:35058, sessionid = 0x13c98509ebf0008, negotiated timeout = 20000
[junit4:junit4]   2> 8910 T827 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@48aa8a97 name:ZooKeeperConnection Watcher:127.0.0.1:35058/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 8911 T771 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 8912 T776 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c98509ebf0008 type:create cxid:0x1 zxid:0x6d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8914 T776 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c98509ebf0008 type:create cxid:0x2 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8916 T771 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38700_
[junit4:junit4]   2> 8917 T776 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c98509ebf0008 type:delete cxid:0x4 zxid:0x6f txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:38700_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:38700_
[junit4:junit4]   2> 8924 T771 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:38700_
[junit4:junit4]   2> 8927 T812 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8928 T793 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 8928 T799 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 8928 T793 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8928 T799 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8931 T812 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 8933 T771 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 8940 T828 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359765094449/collection1
[junit4:junit4]   2> 8941 T828 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 8941 T828 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 8942 T828 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 8943 T828 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359765094449/collection1/'
[junit4:junit4]   2> 8944 T828 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359765094449/collection1/lib/README' to classloader
[junit4:junit4]   2> 8945 T828 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359765094449/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 9000 T828 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 9073 T828 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 9175 T828 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 9182 T828 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 9810 T828 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 9822 T828 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 9826 T828 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 9845 T828 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 9850 T828 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 9855 T828 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 9856 T828 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 9857 T828 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359765094449/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty2/
[junit4:junit4]   2> 9858 T828 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@75fee96
[junit4:junit4]   2> 9858 T828 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 9859 T828 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty2 forceNew:false
[junit4:junit4]   2> 9860 T828 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty2
[junit4:junit4]   2> 9860 T828 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty2/index/
[junit4:junit4]   2> 9861 T828 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 9862 T828 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty2/index forceNew:false
[junit4:junit4]   2> 9873 T828 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@468a8675; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 9874 T828 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 9875 T828 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty2/index
[junit4:junit4]   2> 9875 T828 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 9877 T828 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 9878 T828 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 9878 T828 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 9878 T828 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 9879 T828 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 9879 T828 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 9880 T828 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 9880 T828 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 9881 T828 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 9881 T828 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 9882 T828 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 9883 T828 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 9883 T828 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 9883 T828 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 9884 T828 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 9884 T828 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 9885 T828 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9885 T828 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9886 T828 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9886 T828 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9887 T828 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9887 T828 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9888 T828 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9889 T828 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9889 T828 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 9890 T828 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 9904 T828 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 9909 T828 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 9910 T828 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 9913 T828 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty2
[junit4:junit4]   2> 9915 T828 oass.SolrIndexSearcher.<init> Opening Searcher@4f2ce7fa main
[junit4:junit4]   2> 9916 T828 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 9916 T828 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 9917 T828 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 9926 T828 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 9987 T829 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4f2ce7fa main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 9992 T828 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 9992 T828 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 10056 T794 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 10057 T794 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:35236_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:35236"}
[junit4:junit4]   2> 10062 T794 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:38700_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:38700"}
[junit4:junit4]   2> 10062 T794 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 10062 T794 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 10082 T793 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10082 T812 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10082 T827 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10082 T799 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10995 T828 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 10995 T828 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:38700 collection:collection1 shard:shard2
[junit4:junit4]   2> 10996 T828 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4]   2> 11037 T776 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c98509ebf0008 type:delete cxid:0x42 zxid:0x7d txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
[junit4:junit4]   2> 11038 T828 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 11039 T776 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c98509ebf0008 type:create cxid:0x43 zxid:0x7e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 11050 T828 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 11050 T828 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 11050 T828 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:38700/collection1/
[junit4:junit4]   2> 11051 T828 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 11051 T828 oasc.SyncStrategy.syncToMe http://127.0.0.1:38700/collection1/ has no replicas
[junit4:junit4]   2> 11051 T828 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:38700/collection1/
[junit4:junit4]   2> 11051 T828 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4]   2> 11056 T776 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c98509ebf0008 type:create cxid:0x4c zxid:0x81 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 11590 T794 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11607 T793 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11608 T827 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11608 T812 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11607 T799 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11627 T828 oasc.ZkController.register We are http://127.0.0.1:38700/collection1/ and leader is http://127.0.0.1:38700/collection1/
[junit4:junit4]   2> 11627 T828 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:38700
[junit4:junit4]   2> 11627 T828 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 11627 T828 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 11627 T828 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 11629 T828 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11630 T771 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 11631 T771 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 11631 T771 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 11810 T771 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 11813 T771 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:38710
[junit4:junit4]   2> 11814 T771 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 11814 T771 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 11815 T771 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359765097521
[junit4:junit4]   2> 11815 T771 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359765097521/solr.xml
[junit4:junit4]   2> 11816 T771 oasc.CoreContainer.<init> New CoreContainer 1766128433
[junit4:junit4]   2> 11816 T771 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359765097521/'
[junit4:junit4]   2> 11817 T771 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359765097521/'
[junit4:junit4]   2> 11867 T771 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 11868 T771 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 11868 T771 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 11868 T771 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 11869 T771 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 11869 T771 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 11870 T771 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 11870 T771 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 11870 T771 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 11871 T771 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 11890 T771 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 11908 T771 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:35058/solr
[junit4:junit4]   2> 11909 T771 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 11909 T771 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:35058 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@46679492
[junit4:junit4]   2> 11910 T771 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 11911 T839 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:35058. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 11911 T839 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:35058, initiating session
[junit4:junit4]   2> 11912 T773 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:38711
[junit4:junit4]   2> 11912 T773 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:38711
[junit4:junit4]   2> 11914 T775 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c98509ebf0009 with negotiated timeout 20000 for client /140.211.11.196:38711
[junit4:junit4]   2> 11914 T839 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:35058, sessionid = 0x13c98509ebf0009, negotiated timeout = 20000
[junit4:junit4]   2> 11915 T840 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@46679492 name:ZooKeeperConnection Watcher:127.0.0.1:35058 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 11915 T771 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 11916 T776 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c98509ebf0009
[junit4:junit4]   2> 11926 T840 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 11926 T773 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:38711 which had sessionid 0x13c98509ebf0009
[junit4:junit4]   2> 11926 T771 oaz.ZooKeeper.close Session: 0x13c98509ebf0009 closed
[junit4:junit4]   2> 11926 T771 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 11937 T771 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:35058/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@1bdaaa8e
[junit4:junit4]   2> 11938 T771 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 11938 T841 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:35058. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 11939 T841 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:35058, initiating session
[junit4:junit4]   2> 11939 T773 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:38712
[junit4:junit4]   2> 11939 T773 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:38712
[junit4:junit4]   2> 11940 T775 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c98509ebf000a with negotiated timeout 20000 for client /140.211.11.196:38712
[junit4:junit4]   2> 11940 T841 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:35058, sessionid = 0x13c98509ebf000a, negotiated timeout = 20000
[junit4:junit4]   2> 11941 T842 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1bdaaa8e name:ZooKeeperConnection Watcher:127.0.0.1:35058/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 11941 T771 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 11942 T776 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c98509ebf000a type:create cxid:0x1 zxid:0x8e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 11944 T776 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c98509ebf000a type:create cxid:0x2 zxid:0x8f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 11952 T771 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38710_
[junit4:junit4]   2> 11952 T776 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c98509ebf000a type:delete cxid:0x4 zxid:0x90 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:38710_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:38710_
[junit4:junit4]   2> 11953 T771 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:38710_
[junit4:junit4]   2> 11956 T812 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11957 T793 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 11957 T827 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 11957 T793 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11957 T799 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 11957 T827 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11957 T799 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11958 T812 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 11966 T771 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 11974 T843 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359765097521/collection1
[junit4:junit4]   2> 11974 T843 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 11975 T843 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 11975 T843 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 11976 T843 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359765097521/collection1/'
[junit4:junit4]   2> 11978 T843 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359765097521/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 11978 T843 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359765097521/collection1/lib/README' to classloader
[junit4:junit4]   2> 12032 T843 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 12125 T843 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 12226 T843 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 12247 T843 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 12865 T843 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 12876 T843 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 12880 T843 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 12897 T843 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 12902 T843 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 12907 T843 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 12908 T843 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 12910 T843 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359765097521/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty3/
[junit4:junit4]   2> 12910 T843 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@75fee96
[junit4:junit4]   2> 12910 T843 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 12911 T843 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty3 forceNew:false
[junit4:junit4]   2> 12912 T843 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty3
[junit4:junit4]   2> 12912 T843 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty3/index/
[junit4:junit4]   2> 12913 T843 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 12914 T843 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty3/index forceNew:false
[junit4:junit4]   2> 12921 T843 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1f3a2a97; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 12921 T843 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 12922 T843 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty3/index
[junit4:junit4]   2> 12923 T843 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 12925 T843 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 12925 T843 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 12926 T843 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 12926 T843 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 12927 T843 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 12927 T843 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 12927 T843 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 12928 T843 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 12928 T843 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 12929 T843 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 12930 T843 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 12930 T843 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 12931 T843 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 12931 T843 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 12931 T843 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 12932 T843 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 12932 T843 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12933 T843 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12933 T843 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12934 T843 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12934 T843 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12935 T843 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12936 T843 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12936 T843 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12937 T843 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 12937 T843 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 12950 T843 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 12955 T843 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 12956 T843 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 12958 T843 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty3
[junit4:junit4]   2> 12960 T843 oass.SolrIndexSearcher.<init> Opening Searcher@22d812de main
[junit4:junit4]   2> 12961 T843 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 12961 T843 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 12962 T843 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 13008 T843 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 13055 T844 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@22d812de main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 13060 T843 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 13060 T843 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 13122 T794 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 13122 T794 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:38700_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:38700"}
[junit4:junit4]   2> 13128 T794 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:38710_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:38710"}
[junit4:junit4]   2> 13128 T794 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 13129 T794 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 13139 T842 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13139 T812 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13139 T827 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13139 T799 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13139 T793 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14063 T843 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 14063 T843 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:38710 collection:collection1 shard:shard1
[junit4:junit4]   2> 14067 T843 oasc.ZkController.register We are http://127.0.0.1:38710/collection1/ and leader is http://127.0.0.1:35236/collection1/
[junit4:junit4]   2> 14067 T843 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:38710
[junit4:junit4]   2> 14068 T843 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 14068 T843 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C30 name=collection1 org.apache.solr.core.SolrCore@47defe79 url=http://127.0.0.1:38710/collection1 node=127.0.0.1:38710_ C30_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:38710_, base_url=http://127.0.0.1:38710}
[junit4:junit4]   2> 14076 T845 C30 P38710 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 14077 T843 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14077 T845 C30 P38710 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 14077 T845 C30 P38710 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 14078 T845 C30 P38710 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 14078 T771 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 14078 T771 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 14079 T771 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 14079 T845 C30 P38710 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 14296 T771 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 14300 T771 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:38718
[junit4:junit4]   2> 14300 T771 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 14301 T771 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 14301 T771 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359765099973
[junit4:junit4]   2> 14302 T771 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359765099973/solr.xml
[junit4:junit4]   2> 14302 T771 oasc.CoreContainer.<init> New CoreContainer 204378465
[junit4:junit4]   2> 14303 T771 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359765099973/'
[junit4:junit4]   2> 14303 T771 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359765099973/'
[junit4:junit4]   2> 14366 T771 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 14366 T771 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 14367 T771 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 14367 T771 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 14368 T771 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 14368 T771 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 14368 T771 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 14369 T771 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 14369 T771 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 14370 T771 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 14393 T771 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 14414 T771 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:35058/solr
[junit4:junit4]   2> 14414 T771 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 14415 T771 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:35058 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@3d954a50
[junit4:junit4]   2> 14416 T771 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 14417 T855 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:35058. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 14418 T855 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:35058, initiating session
[junit4:junit4]   2> 14418 T773 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:38719
[junit4:junit4]   2> 14418 T773 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:38719
[junit4:junit4]   2> 14420 T775 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c98509ebf000b with negotiated timeout 20000 for client /140.211.11.196:38719
[junit4:junit4]   2> 14420 T855 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:35058, sessionid = 0x13c98509ebf000b, negotiated timeout = 20000
[junit4:junit4]   2> 14420 T856 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3d954a50 name:ZooKeeperConnection Watcher:127.0.0.1:35058 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 14421 T771 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 14422 T776 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c98509ebf000b
[junit4:junit4]   2> 14423 T856 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 14423 T773 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:38719 which had sessionid 0x13c98509ebf000b
[junit4:junit4]   2> 14423 T771 oaz.ZooKeeper.close Session: 0x13c98509ebf000b closed
[junit4:junit4]   2> 14424 T771 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 14436 T771 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:35058/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@7d680c83
[junit4:junit4]   2> 14437 T771 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 14437 T857 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:35058. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 14438 T857 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:35058, initiating session
[junit4:junit4]   2> 14438 T773 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:38720
[junit4:junit4]   2> 14438 T773 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:38720
[junit4:junit4]   2> 14440 T775 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c98509ebf000c with negotiated timeout 20000 for client /140.211.11.196:38720
[junit4:junit4]   2> 14440 T857 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:35058, sessionid = 0x13c98509ebf000c, negotiated timeout = 20000
[junit4:junit4]   2> 14440 T858 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7d680c83 name:ZooKeeperConnection Watcher:127.0.0.1:35058/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 14440 T771 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 14442 T776 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c98509ebf000c type:create cxid:0x1 zxid:0xa0 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 14443 T776 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c98509ebf000c type:create cxid:0x2 zxid:0xa1 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 14451 T771 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38718_
[junit4:junit4]   2> 14451 T776 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c98509ebf000c type:delete cxid:0x4 zxid:0xa2 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:38718_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:38718_
[junit4:junit4]   2> 14452 T771 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:38718_
[junit4:junit4]   2> 14455 T812 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14456 T793 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 14456 T842 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 14456 T793 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14456 T842 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14456 T827 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 14456 T799 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 14457 T799 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14457 T827 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14458 T812 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 14460 T771 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 14468 T859 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359765099973/collection1
[junit4:junit4]   2> 14469 T859 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 14469 T859 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 14470 T859 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 14471 T859 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359765099973/collection1/'
[junit4:junit4]   2> 14473 T859 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359765099973/collection1/lib/README' to classloader
[junit4:junit4]   2> 14473 T859 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359765099973/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 14544 T859 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 14620 T859 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 14653 T794 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14653 T794 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:38710_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:38710"}
[junit4:junit4]   2> 14666 T793 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14666 T799 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14666 T827 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14666 T842 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14666 T812 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14721 T859 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 14721 T858 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14729 T859 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 15333 T859 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 15344 T859 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 15348 T859 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 15365 T859 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 15370 T859 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 15375 T859 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 15376 T859 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 15378 T859 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359765099973/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty4/
[junit4:junit4]   2> 15378 T859 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@75fee96
[junit4:junit4]   2> 15379 T859 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 15379 T859 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty4 forceNew:false
[junit4:junit4]   2> 15380 T859 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty4
[junit4:junit4]   2> 15380 T859 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty4/index/
[junit4:junit4]   2> 15381 T859 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 15382 T859 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty4/index forceNew:false
[junit4:junit4]   2> 15413 T859 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4a8b9b70; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 15414 T859 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 15415 T859 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty4/index
[junit4:junit4]   2> 15415 T859 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 15417 T859 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 15417 T859 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 15418 T859 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 15418 T859 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 15419 T859 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 15419 T859 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 15419 T859 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 15420 T859 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 15421 T859 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 15421 T859 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 15422 T859 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 15422 T859 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 15423 T859 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 15423 T859 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 15423 T859 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 15424 T859 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 15425 T859 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15425 T859 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15426 T859 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15426 T859 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15427 T859 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15427 T859 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15428 T859 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15428 T859 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 15429 T859 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 15430 T859 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 15442 T859 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 15447 T859 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 15448 T859 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 15451 T859 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty4
[junit4:junit4]   2> 15453 T859 oass.SolrIndexSearcher.<init> Opening Searcher@75b86d67 main
[junit4:junit4]   2> 15454 T859 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 15454 T859 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 15454 T859 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 15462 T859 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 15514 T860 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@75b86d67 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 15519 T859 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 15519 T859 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 16172 T794 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16173 T794 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:38718_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:38718"}
[junit4:junit4]   2> 16173 T794 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 16173 T794 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 16216 T858 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16216 T793 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16216 T812 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16216 T799 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16216 T827 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16216 T842 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16522 T859 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 16522 T859 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:38718 collection:collection1 shard:shard2
[junit4:junit4]   2> 16527 T859 oasc.ZkController.register We are http://127.0.0.1:38718/collection1/ and leader is http://127.0.0.1:38700/collection1/
[junit4:junit4]   2> 16527 T859 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:38718
[junit4:junit4]   2> 16528 T859 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 16528 T859 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C31 name=collection1 org.apache.solr.core.SolrCore@7345a749 url=http://127.0.0.1:38718/collection1 node=127.0.0.1:38718_ C31_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:38718_, base_url=http://127.0.0.1:38718}
[junit4:junit4]   2> 16528 T861 C31 P38718 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 16529 T859 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16529 T861 C31 P38718 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 16530 T861 C31 P38718 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 16530 T861 C31 P38718 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 16530 T771 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 16531 T771 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 16532 T771 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 16540 T861 C31 P38718 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 16566 T771 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16568 T771 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4]   2> 16569 T771 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C30_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:38710_, base_url=http://127.0.0.1:38710}
[junit4:junit4]   2> 17123 T845 C30 P38710 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:35236/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 17134 T845 C30 P38710 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4]   2> 17155 T845 C30 P38710 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:38710 START replicas=[http://127.0.0.1:35236/collection1/] nUpdates=100
[junit4:junit4]   2> 17155 T845 C30 P38710 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 17156 T845 C30 P38710 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 17157 T845 C30 P38710 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 17157 T845 C30 P38710 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 17157 T845 C30 P38710 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 17157 T845 C30 P38710 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:35236/collection1/. core=collection1
[junit4:junit4]   2> 17157 T845 C30 P38710 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C32 name=collection1 org.apache.solr.core.SolrCore@4baafcf7 url=http://127.0.0.1:35236/collection1 node=127.0.0.1:35236_ C32_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:35236_, base_url=http://127.0.0.1:35236, leader=true}
[junit4:junit4]   2> 17169 T803 C32 P35236 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 17171 T804 C32 P35236 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 17173 T804 C32 P35236 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty1
[junit4:junit4]   2> 17177 T804 C32 P35236 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@cdcf3d9; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 17178 T804 C32 P35236 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 17211 T804 C32 P35236 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@cdcf3d9; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@cdcf3d9; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 17212 T804 C32 P35236 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 17213 T804 C32 P35236 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty1
[junit4:junit4]   2> 17214 T804 C32 P35236 oass.SolrIndexSearcher.<init> Opening Searcher@40401ab2 realtime
[junit4:junit4]   2> 17214 T804 C32 P35236 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 17215 T804 C32 P35236 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 44
[junit4:junit4]   2> 17216 T845 C30 P38710 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 17216 T845 C30 P38710 oascsi.HttpClientUtil.createClient Creating new http client, config:connTimeout=5000&socketTimeout=20000&allowCompression=false&maxConnections=10000&maxConnectionsPerHost=10000
[junit4:junit4]   2> 17246 T845 C30 P38710 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 17249 T805 C32 P35236 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 17249 T805 C32 P35236 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 17250 T845 C30 P38710 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 17250 T845 C30 P38710 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 17250 T845 C30 P38710 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 17253 T805 C32 P35236 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty1
[junit4:junit4]   2> 17254 T805 C32 P35236 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty1/index
[junit4:junit4]   2> 17255 T805 C32 P35236 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=2 
[junit4:junit4]   2> 17256 T845 C30 P38710 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 17257 T845 C30 P38710 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty3/index.20130202023143131 forceNew:false
[junit4:junit4]   2> 17258 T845 C30 P38710 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty3
[junit4:junit4]   2> 17259 T845 C30 P38710 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty3/index.20130202023143131 lockFactory=org.apache.lucene.store.NativeFSLockFactory@203de59a; maxCacheMB=48.0 maxMergeSizeMB=4.0)) fullCopy=true
[junit4:junit4]   2> 17274 T805 C32 P35236 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=6 
[junit4:junit4]   2> 17288 T845 C30 P38710 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty3/index
[junit4:junit4]   2> 17289 T845 C30 P38710 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 17290 T845 C30 P38710 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130202023143131
[junit4:junit4]   2> 17292 T845 C30 P38710 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty3
[junit4:junit4]   2> 17293 T845 C30 P38710 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty3
[junit4:junit4]   2> 17293 T845 C30 P38710 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty3/index/ new=./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty3/index.20130202023143131
[junit4:junit4]   2> 17297 T845 C30 P38710 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty3/index.20130202023143131 lockFactory=org.apache.lucene.store.NativeFSLockFactory@203de59a; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 17297 T845 C30 P38710 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 17299 T845 C30 P38710 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty3
[junit4:junit4]   2> 17304 T845 C30 P38710 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 17304 T845 C30 P38710 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 17304 T845 C30 P38710 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 17305 T845 C30 P38710 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty3/index.20130202023143131
[junit4:junit4]   2> 17306 T845 C30 P38710 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty3
[junit4:junit4]   2> 17310 T845 C30 P38710 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty3/index.20130202023143131 lockFactory=org.apache.lucene.store.NativeFSLockFactory@203de59a; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 17310 T845 C30 P38710 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 17310 T845 C30 P38710 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 17311 T845 C30 P38710 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty3
[junit4:junit4]   2> 17311 T845 C30 P38710 oass.SolrIndexSearcher.<init> Opening Searcher@79f32b0e main
[junit4:junit4]   2> 17313 T844 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@79f32b0e main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 17314 T844 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty3/index
[junit4:junit4]   2> 17314 T845 C30 P38710 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 17317 T845 C30 P38710 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty3/index.20130202023143131 lockFactory=org.apache.lucene.store.NativeFSLockFactory@203de59a; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty3/index.20130202023143131 lockFactory=org.apache.lucene.store.NativeFSLockFactory@203de59a; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 17318 T845 C30 P38710 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 17319 T845 C30 P38710 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty3
[junit4:junit4]   2> 17319 T845 C30 P38710 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 17319 T845 C30 P38710 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty3/index.20130202023143131
[junit4:junit4]   2> 17320 T845 C30 P38710 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty3/index
[junit4:junit4]   2> 17320 T845 C30 P38710 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 17320 T845 C30 P38710 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 17320 T845 C30 P38710 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 17320 T845 C30 P38710 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 17327 T845 C30 P38710 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 17572 T771 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17722 T794 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17723 T794 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:38718_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:38718"}
[junit4:junit4]   2> 17733 T794 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:38710_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:38710"}
[junit4:junit4]   2> 17744 T793 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17744 T812 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17744 T858 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17744 T799 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17744 T827 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17744 T842 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18575 T771 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19578 T771 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C31_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:38718_, base_url=http://127.0.0.1:38718}
[junit4:junit4]   2> 20575 T861 C31 P38718 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:38700/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 20575 T861 C31 P38718 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:38718 START replicas=[http://127.0.0.1:38700/collection1/] nUpdates=100
[junit4:junit4]   2> 20576 T861 C31 P38718 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 20576 T861 C31 P38718 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 20576 T861 C31 P38718 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 20576 T861 C31 P38718 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 20577 T861 C31 P38718 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 20577 T861 C31 P38718 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:38700/collection1/. core=collection1
[junit4:junit4]   2> 20578 T861 C31 P38718 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 20581 T771 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> ASYNC  NEW_CORE C33 name=collection1 org.apache.solr.core.SolrCore@129f61fd url=http://127.0.0.1:38700/collection1 node=127.0.0.1:38700_ C33_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:38700_, base_url=http://127.0.0.1:38700, leader=true}
[junit4:junit4]   2> 20591 T818 C33 P38700 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1 
[junit4:junit4]   2> 20601 T819 C33 P38700 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 20602 T819 C33 P38700 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty2
[junit4:junit4]   2> 20607 T819 C33 P38700 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@468a8675; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 20608 T819 C33 P38700 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 20612 T819 C33 P38700 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@468a8675; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@468a8675; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 20613 T819 C33 P38700 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 20614 T819 C33 P38700 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty2
[junit4:junit4]   2> 20614 T819 C33 P38700 oass.SolrIndexSearcher.<init> Opening Searcher@48521e1a realtime
[junit4:junit4]   2> 20615 T819 C33 P38700 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 20615 T819 C33 P38700 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 14
[junit4:junit4]   2> 20616 T861 C31 P38718 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 20617 T861 C31 P38718 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 20620 T820 C33 P38700 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 20621 T820 C33 P38700 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 20622 T861 C31 P38718 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 20622 T861 C31 P38718 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 20623 T861 C31 P38718 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 20626 T820 C33 P38700 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty2
[junit4:junit4]   2> 20626 T820 C33 P38700 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty2/index
[junit4:junit4]   2> 20627 T820 C33 P38700 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=2 
[junit4:junit4]   2> 20628 T861 C31 P38718 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 20629 T861 C31 P38718 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty4/index.20130202023146503 forceNew:false
[junit4:junit4]   2> 20629 T861 C31 P38718 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty4
[junit4:junit4]   2> 20630 T861 C31 P38718 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty4/index.20130202023146503 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7a0c5fb2; maxCacheMB=48.0 maxMergeSizeMB=4.0)) fullCopy=true
[junit4:junit4]   2> 20634 T820 C33 P38700 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> 20636 T861 C31 P38718 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty4/index
[junit4:junit4]   2> 20636 T861 C31 P38718 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 20637 T861 C31 P38718 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130202023146503
[junit4:junit4]   2> 20639 T861 C31 P38718 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty4
[junit4:junit4]   2> 20640 T861 C31 P38718 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty4
[junit4:junit4]   2> 20640 T861 C31 P38718 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty4/index/ new=./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty4/index.20130202023146503
[junit4:junit4]   2> 20644 T861 C31 P38718 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty4/index.20130202023146503 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7a0c5fb2; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 20644 T861 C31 P38718 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 20646 T861 C31 P38718 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty4
[junit4:junit4]   2> 20646 T861 C31 P38718 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 20646 T861 C31 P38718 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 20646 T861 C31 P38718 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 20647 T861 C31 P38718 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty4/index.20130202023146503
[junit4:junit4]   2> 20649 T861 C31 P38718 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty4
[junit4:junit4]   2> 20653 T861 C31 P38718 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty4/index.20130202023146503 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7a0c5fb2; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 20653 T861 C31 P38718 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 20654 T861 C31 P38718 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 20654 T861 C31 P38718 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty4
[junit4:junit4]   2> 20655 T861 C31 P38718 oass.SolrIndexSearcher.<init> Opening Searcher@67bd5b5b main
[junit4:junit4]   2> 20656 T860 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@67bd5b5b main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 20657 T860 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty4/index
[junit4:junit4]   2> 20657 T861 C31 P38718 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 20661 T861 C31 P38718 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty4/index.20130202023146503 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7a0c5fb2; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty4/index.20130202023146503 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7a0c5fb2; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 20661 T861 C31 P38718 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 20662 T861 C31 P38718 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty4
[junit4:junit4]   2> 20663 T861 C31 P38718 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 20663 T861 C31 P38718 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty4/index.20130202023146503
[junit4:junit4]   2> 20663 T861 C31 P38718 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty4/index
[junit4:junit4]   2> 20663 T861 C31 P38718 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 20663 T861 C31 P38718 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 20664 T861 C31 P38718 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 20664 T861 C31 P38718 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 20671 T861 C31 P38718 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 20770 T794 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 20771 T794 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:38718_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:38718"}
[junit4:junit4]   2> 20788 T793 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20788 T858 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20788 T827 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20788 T799 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20788 T842 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20788 T812 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 21585 T771 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 21586 T771 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4]   2> ASYNC  NEW_CORE C34 name=collection1 org.apache.solr.core.SolrCore@39f48c45 url=http://127.0.0.1:65305/collection1 node=127.0.0.1:65305_ C34_STATE=coll:control_collection core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:65305_, base_url=http://127.0.0.1:65305, leader=true}
[junit4:junit4]   2> 21600 T783 C34 P65305 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/control/data
[junit4:junit4]   2> 21605 T783 C34 P65305 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@20ffca31; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 21605 T783 C34 P65305 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 21607 T783 C34 P65305 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/control/data
[junit4:junit4]   2> 21608 T783 C34 P65305 oass.SolrIndexSearcher.<init> Opening Searcher@7a2deefb realtime
[junit4:junit4]   2> 21608 T783 C34 P65305 /update {wt=javabin&version=2} {deleteByQuery=*:* (-1425817057334657024)} 0 10
[junit4:junit4]   2> 21614 T806 C32 P35236 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty1
[junit4:junit4]   2> 21621 T821 C33 P38700 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty2
[junit4:junit4]   2>  C30_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:38710_, base_url=http://127.0.0.1:38710}
[junit4:junit4]   2> 21628 T832 C30 P38710 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty3
[junit4:junit4]   2> 21629 T832 C30 P38710 /update {update.distrib=FROMLEADER&_version_=-1425817057348288512&update.from=http://127.0.0.1:35236/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1425817057348288512)} 0 3
[junit4:junit4]   2>  C31_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:38718_, base_url=http://127.0.0.1:38718}
[junit4:junit4]   2> 21634 T848 C31 P38718 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359765085881/jetty4
[junit4:junit4]   2> 21635 T848 C31 P38718 /update {update.distrib=FROMLEADER&_version_=-1425817057355628544&update.from=http://127.0.0.1:38700/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1425817057355628544)} 0 3
[junit4:junit4]   2> 21636 T821 C33 P38700 /update {update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1425817057355628544)} 0 17
[junit4:junit4]   2> 21637 T806 C32 P35236 /update {wt=javabin&version=2} {deleteByQuery=*:* (-1425817057348288512)} 0 26
[junit4:junit4]   2> 21646 T783 C34 P65305 /update {wt=javabin&version=2} {add=[1 (1425817057379745792)]} 0 4
[junit4:junit4]   2> 21659 T832 C30 P38710 /update {distrib.from=http://127.0.0.1:35236/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1425817057388134400)]} 0 4
[junit4:junit4]   2> 21660 T806 C32 P35236 /update {wt=javabin&version=2} {add=[1 (1425817057388134400)]} 0 11
[junit4:junit4]   2> 21665 T783 C34 P65305 /update {wt=javabin&version=2} {add=[2 (1425817057402814464)]} 0 2
[junit4:junit4]   2> 21686 T848 C31 P38718 /update {distrib.from=http://127.0.0.1:38700/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1425817057416445952)]} 0 3
[junit4:junit4]   2> 21687 T821 C33 P38700 /update {distrib.from=http://127.0.0.1:35236/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[2 (1425817057416445952)]} 0 10
[junit4:junit4]   2> 21688 T806 C32 P35236 /update {wt=javabin&version=2} {add=[2]} 0 20
[junit4:junit4]   2> 21693 T783 C34 P65305 /update {wt=javabin&version=2} {add=[3 (1425817057432174592)]} 0 1
[junit4:junit4]   2> 21703 T848 C31 P38718 /update {distrib.from=http://127.0.0.1:38700/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[3 (1425817057436368896)]} 0 2
[junit4:junit4]   2> 21704 T822 C33 P38700 /update {wt=javabin&version=2} {add=[3 (1425817057436368896)]} 0 8
[junit4:junit4]   2> 21709 T783 C34 P65305 /update {wt=javabin&version=2} {add=[4 (1425817057448951808)]} 0 2
[junit4:junit4]   2> 21718 T832 C30 P38710 /update {distrib.from=http://127.0.0.1:35236/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[4 (1425817057454194688)]} 0 2
[junit4:junit4]   2> 21719 T806 C32 P35236 /update {wt=javabin&version=2} {add=[4 (1425817057454194688)]} 0 7
[junit4:junit4]   2> 21724 T783 C34 P65305 /update {wt=javabin&version=2} {add=[5 (1425817057464680448)]} 0 2
[junit4:junit4]   2> 21733 T848 C31 P38718 /update {distrib.from=http://127.0.0.1:38700/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[5 (1425817057469923328)]} 0 2
[junit4:junit4]   2> 21734 T822 C33 P38700 /update {wt=javabin&version=2} {add=[5 (1425817057469923328)]} 0 7
[junit4:junit4]   2> 21739 T783 C34 P65305 /update {wt=javabin&version=2} {add=[6 (1425817057480409088)]} 0 2
[junit4:junit4]   2> 21751 T848 C31 P38718 /update {distrib.from=http://127.0.0.1:38700/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[6 (1425817057488797696)]} 0 2
[junit4:junit4]   2> 21752 T821 C33 P38700 /update {distrib.from=http://127.0.0.1:35236/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[6 (1425817057488797696)]} 0 7
[junit4:junit4]   2> 21753 T806 C32 P35236 /update {wt=javabin&version=2} {add=[6]} 0 11
[junit4:junit4]   2> 21758 T783 C34 P65305 /update {wt=javabin&version=2} {add=[7 (1425817057500332032)]} 0 2
[junit4:junit4]   2> 21767 T848 C31 P38718 /update {distrib.from=http://127.0.0.1:38700/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[7 (1425817057505574912)]} 0 1
[junit4:junit4]   2> 21769 T822 C33 P38700 /update {wt=javabin&version=2} {add=[7 (1425817057505574912)]} 0 8
[junit4:junit4]   2> 21775 T783 C34 P65305 /update {wt=javabin&version=2} {add=[8 (1425817057517109248)]} 0 3
[junit4:junit4]   2> 22053 T832 C30 P38710 /update {distrib.from=http://127.0.0.1:35236/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[8 (1425817057530740736)]} 0 2
[junit4:junit4]   2> 22055 T807 C32 P35236 /update {distrib.from=http://127.0.0.1:38700/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[8 (1425817057530740736)]} 0 270
[junit4:junit4]   2> 22056 T822 C33 P38700 /update {wt=javabin&version=2} {add=[8]} 0 276
[junit4:junit4]   2> 22062 T783 C34 P65305 /update {wt=javabin&version=2} {add=[9 (1425817057818050560)]} 0 3
[junit4:junit4]   2> 22077 T848 C31 P38718 /update {distrib.from=http://127.0.0.1:38700/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[9 (1425817057829584896)]} 0 2
[junit4:junit4]   2> 22078 T821 C33 P38700 /update {distrib.from=http://127.0.0.1:35236/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[9 (1425817057829584896)]} 0 8
[junit4:junit4]   2> 22079 T806 C32 P35236 /update {wt=javabin&version=2} {add=[9]} 0 13
[junit4:junit4]   2> 22104 T783 C34 P65305 /update {wt=javabin&version=2} {add=[10 (1425817057843216384)]} 0 21
[junit4:junit4]   2> 22115 T832 C30 P38710 /update {distrib.from=http://127.0.0.1:35236/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10 (1425817057869430784)]} 0 2
[junit4:junit4]   2> 22116 T806 C32 P35236 /update {wt=javabin&version=2} {add=[10 (1425817057869430784)]} 0 8
[junit4:junit4]   2> 22124 T783 C34 P65305 /update {wt=javabin&version=2} {add=[11 (1425817057882013696)]} 0 4
[junit4:junit4]   2> 22137 T832 C30 P38710 /update {distrib.from=http://127.0.0.1:35236/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[11 (1425817057892499456)]} 0 2
[junit4:junit4]   2> 22138 T807 C32 P35236 /update {distrib.from=http://127.0.0.1:38700/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[11 (1425817057892499456)]} 0 8
[junit4:junit4]   2> 22139 T822 C33 P38700 /update {wt=javabin&version=2} {add=[11]} 0 12
[junit4:junit4]   2> 22145 T783 C34 P65305 /update {wt=javabin&version=2} {add=[12 (1425817057906130944)]} 0 2
[junit4:junit4]   2> 22154 T832 C30 P38710 /update {distrib.from=http://127.0.0.1:35236/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[12 (1425817057911373824)]} 0 2
[junit4:junit4]   2> 22155 T806 C32 P35236 /update {wt=javabin&version=2} {add=[12 (1425817057911373824)]} 0 7
[junit4:junit4]   2> 22162 T783 C34 P65305 /update {wt=javabin&version=2} {add=[13 (1425817057922908160)]} 0 3
[junit4:junit4]   2> 22175 T832 C30 P38710 /update {distrib.from=http://127.0.0.1:35236/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[13 (1425817057932345344)]} 0 2
[junit4:junit4]   2> 22177 T807 C32 P35236 /update {dist

[...truncated too long message...]

getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  25) Thread[id=1277, name=RecoveryThread, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at java.net.SocketInputStream.socketRead0(Native Method)
[junit4:junit4]   2> 	        at java.net.SocketInputStream.read(SocketInputStream.java:150)
[junit4:junit4]   2> 	        at java.net.SocketInputStream.read(SocketInputStream.java:121)
[junit4:junit4]   2> 	        at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:149)
[junit4:junit4]   2> 	        at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:111)
[junit4:junit4]   2> 	        at org.apache.http.impl.io.SocketInputBuffer.isDataAvailable(SocketInputBuffer.java:122)
[junit4:junit4]   2> 	        at org.apache.http.impl.AbstractHttpClientConnection.isStale(AbstractHttpClientConnection.java:311)
[junit4:junit4]   2> 	        at org.apache.http.impl.conn.AbstractClientConnAdapter.isStale(AbstractClientConnAdapter.java:170)
[junit4:junit4]   2> 	        at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:412)
[junit4:junit4]   2> 	        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
[junit4:junit4]   2> 	        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
[junit4:junit4]   2> 	        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:732)
[junit4:junit4]   2> 	        at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:353)
[junit4:junit4]   2> 	        at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181)
[junit4:junit4]   2> 	        at org.apache.solr.handler.SnapPuller.fetchFileList(SnapPuller.java:268)
[junit4:junit4]   2> 	        at org.apache.solr.handler.SnapPuller.fetchLatestIndex(SnapPuller.java:362)
[junit4:junit4]   2> 	        at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:275)
[junit4:junit4]   2> 	        at org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:153)
[junit4:junit4]   2> 	        at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:409)
[junit4:junit4]   2> 	        at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:223)
[junit4:junit4]   2> 	  26) Thread[id=838, name=qtp1903199973-838, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at java.net.SocketInputStream.socketRead0(Native Method)
[junit4:junit4]   2> 	        at java.net.SocketInputStream.read(SocketInputStream.java:150)
[junit4:junit4]   2> 	        at java.net.SocketInputStream.read(SocketInputStream.java:121)
[junit4:junit4]   2> 	        at org.eclipse.jetty.io.ByteArrayBuffer.readFrom(ByteArrayBuffer.java:375)
[junit4:junit4]   2> 	        at org.eclipse.jetty.io.bio.StreamEndPoint.fill(StreamEndPoint.java:141)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.fill(SocketConnector.java:227)
[junit4:junit4]   2> 	        at org.eclipse.jetty.http.HttpParser.fill(HttpParser.java:1035)
[junit4:junit4]   2> 	        at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:280)
[junit4:junit4]   2> 	        at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  27) Thread[id=847, name=qtp1694833632-847 Acceptor0 SocketConnector@127.0.0.1:38718, state=RUNNABLE, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at java.net.PlainSocketImpl.socketAccept(Native Method)
[junit4:junit4]   2> 	        at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
[junit4:junit4]   2> 	        at java.net.ServerSocket.implAccept(ServerSocket.java:522)
[junit4:junit4]   2> 	        at java.net.ServerSocket.accept(ServerSocket.java:490)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.bio.SocketConnector.accept(SocketConnector.java:117)
[junit4:junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
[junit4:junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> NOTE: test params are: codec=Lucene42: {n_f1=Pulsing41(freqCutoff=14 minBlockSize=67 maxBlockSize=221), foo_b=MockFixedIntBlock(blockSize=1405), cat=PostingsFormat(name=MockRandom), foo_d=PostingsFormat(name=SimpleText), foo_f=MockFixedIntBlock(blockSize=1405), n_tl1=Pulsing41(freqCutoff=14 minBlockSize=67 maxBlockSize=221), n_d1=PostingsFormat(name=MockRandom), rnd_b=PostingsFormat(name=SimpleText), intDefault=PostingsFormat(name=MockRandom), n_td1=Pulsing41(freqCutoff=14 minBlockSize=67 maxBlockSize=221), timestamp=PostingsFormat(name=MockRandom), id=PostingsFormat(name=SimpleText), range_facet_sl=MockFixedIntBlock(blockSize=1405), range_facet_si=Pulsing41(freqCutoff=14 minBlockSize=67 maxBlockSize=221), oddField_s=MockFixedIntBlock(blockSize=1405), sequence_i=MockFixedIntBlock(blockSize=1405), name=PostingsFormat(name=SimpleText), foo_i=Pulsing41(freqCutoff=14 minBlockSize=67 maxBlockSize=221), regex_dup_B_s=PostingsFormat(name=MockRandom), multiDefault=Pulsing41(freqCutoff=14 minBlockSize=67 maxBlockSize=221), n_tf1=PostingsFormat(name=MockRandom), n_dt1=Pulsing41(freqCutoff=14 minBlockSize=67 maxBlockSize=221), genre_s=MockFixedIntBlock(blockSize=1405), author_t=Pulsing41(freqCutoff=14 minBlockSize=67 maxBlockSize=221), n_ti1=PostingsFormat(name=SimpleText), range_facet_l=PostingsFormat(name=MockRandom), text=MockFixedIntBlock(blockSize=1405), _version_=PostingsFormat(name=MockRandom), val_i=PostingsFormat(name=SimpleText), SubjectTerms_mfacet=PostingsFormat(name=MockRandom), series_t=Pulsing41(freqCutoff=14 minBlockSize=67 maxBlockSize=221), a_t=PostingsFormat(name=MockRandom), n_tdt1=PostingsFormat(name=MockRandom), regex_dup_A_s=MockFixedIntBlock(blockSize=1405), price=MockFixedIntBlock(blockSize=1405), other_tl1=PostingsFormat(name=MockRandom), n_l1=PostingsFormat(name=MockRandom), a_si=Pulsing41(freqCutoff=14 minBlockSize=67 maxBlockSize=221), inStock=MockFixedIntBlock(blockSize=1405)}, sim=DefaultSimilarity, locale=en_GB, timezone=Libya
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=28,free=129210448,total=363855872
[junit4:junit4]   2> NOTE: All tests run in this JVM: [SolrCmdDistributorTest, JSONWriterTest, DirectUpdateHandlerOptimizeTest, TestFastOutputStream, AnalysisAfterCoreReloadTest, FieldAnalysisRequestHandlerTest, TestPseudoReturnFields, SimpleFacetsTest, CircularListTest, TestBinaryResponseWriter, MBeansHandlerTest, DistributedTermsComponentTest, DateFieldTest, TestIndexSearcher, TestConfig, ZkControllerTest, TestSolrQueryParser, TestBadConfig, SearchHandlerTest, SpellPossibilityIteratorTest, SolrCoreCheckLockOnStartupTest, TestDefaultSimilarityFactory, TestPerFieldSimilarity, TestRandomFaceting, ClusterStateUpdateTest, ResourceLoaderTest, NumericFieldsTest, AlternateDirectoryTest, PreAnalyzedFieldTest, TestPHPSerializedResponseWriter, TestJoin, ZkCLITest, TestLazyCores, BasicDistributedZkTest]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=BasicDistributedZkTest -Dtests.seed=5CD2923B7FED216C -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=en_GB -Dtests.timezone=Libya -Dtests.file.encoding=US-ASCII
[junit4:junit4] ERROR   0.00s J0 | BasicDistributedZkTest (suite) <<<
[junit4:junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([5CD2923B7FED216C]:0)
[junit4:junit4] Completed on J0 in 7223.67s, 1 test, 2 errors <<< FAILURES!

[...truncated 89 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:346: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/common-build.xml:428: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:1176: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:841: There were test failures: 249 suites, 1038 tests, 1 suite-level error, 1 error, 111 ignored (4 assumptions)

Total time: 146 minutes 51 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-trunk-java7 - Build # 3699 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-java7/3699/

1 tests failed.
REGRESSION:  org.apache.solr.cloud.SyncSliceTest.testDistribSearch

Error Message:
shard1 should have just been set up to be inconsistent - but it's still consistent

Stack Trace:
java.lang.AssertionError: shard1 should have just been set up to be inconsistent - but it's still consistent
	at __randomizedtesting.SeedInfo.seed([F7EF88BAABE35641:760906A2DCBC367D]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNotNull(Assert.java:526)
	at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:214)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:794)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:722)




Build Log:
[...truncated 9411 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.SyncSliceTest
[junit4:junit4]   2> 0 T1932 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /r_ziz/a
[junit4:junit4]   2> 5 T1932 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./solrtest-SyncSliceTest-1359752327694
[junit4:junit4]   2> 6 T1932 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 7 T1933 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 10 T1933 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 10 T1933 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4]   2> 10 T1933 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4]   2> 11 T1933 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 11 T1933 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4]   2> 108 T1932 oasc.ZkTestServer.run start zk server on port:18314
[junit4:junit4]   2> 108 T1932 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:18314 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@56a085ff
[junit4:junit4]   2> 109 T1932 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 109 T1938 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:18314. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 110 T1938 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:18314, initiating session
[junit4:junit4]   2> 110 T1934 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:18315
[junit4:junit4]   2> 111 T1934 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:18315
[junit4:junit4]   2> 111 T1936 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 114 T1936 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c978df2140000 with negotiated timeout 10000 for client /140.211.11.196:18315
[junit4:junit4]   2> 114 T1938 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:18314, sessionid = 0x13c978df2140000, negotiated timeout = 10000
[junit4:junit4]   2> 114 T1939 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@56a085ff name:ZooKeeperConnection Watcher:127.0.0.1:18314 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 114 T1932 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 115 T1932 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 118 T1937 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c978df2140000
[junit4:junit4]   2> 124 T1939 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 124 T1932 oaz.ZooKeeper.close Session: 0x13c978df2140000 closed
[junit4:junit4]   2> 125 T1934 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:18315 which had sessionid 0x13c978df2140000
[junit4:junit4]   2> 125 T1932 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:18314/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@17849fe7
[junit4:junit4]   2> 126 T1932 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 126 T1940 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:18314. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 127 T1940 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:18314, initiating session
[junit4:junit4]   2> 127 T1934 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:18316
[junit4:junit4]   2> 127 T1934 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:18316
[junit4:junit4]   2> 128 T1936 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c978df2140001 with negotiated timeout 10000 for client /140.211.11.196:18316
[junit4:junit4]   2> 128 T1940 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:18314, sessionid = 0x13c978df2140001, negotiated timeout = 10000
[junit4:junit4]   2> 129 T1941 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@17849fe7 name:ZooKeeperConnection Watcher:127.0.0.1:18314/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 129 T1932 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 129 T1932 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 140 T1932 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 142 T1932 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 145 T1932 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 147 T1932 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 148 T1932 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 161 T1932 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 162 T1932 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 266 T1932 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 267 T1932 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 270 T1932 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 271 T1932 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 281 T1932 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 281 T1932 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 285 T1932 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 285 T1932 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 288 T1932 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 289 T1932 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 292 T1932 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 293 T1932 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 296 T1932 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 297 T1932 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 300 T1937 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c978df2140001
[junit4:junit4]   2> 307 T1941 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 307 T1932 oaz.ZooKeeper.close Session: 0x13c978df2140001 closed
[junit4:junit4]   2> 307 T1934 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:18316 which had sessionid 0x13c978df2140001
[junit4:junit4]   2> 475 T1932 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 480 T1932 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:18317
[junit4:junit4]   2> 481 T1932 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 482 T1932 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 482 T1932 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1359752327996
[junit4:junit4]   2> 482 T1932 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1359752327996/solr.xml
[junit4:junit4]   2> 483 T1932 oasc.CoreContainer.<init> New CoreContainer 14269640
[junit4:junit4]   2> 484 T1932 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1359752327996/'
[junit4:junit4]   2> 484 T1932 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1359752327996/'
[junit4:junit4]   2> 539 T1932 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 540 T1932 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 540 T1932 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 541 T1932 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 541 T1932 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 542 T1932 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 542 T1932 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 543 T1932 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 543 T1932 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 544 T1932 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 566 T1932 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 586 T1932 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:18314/solr
[junit4:junit4]   2> 586 T1932 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 587 T1932 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:18314 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@2c45fe62
[junit4:junit4]   2> 588 T1932 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 588 T1951 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:18314. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 589 T1951 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:18314, initiating session
[junit4:junit4]   2> 589 T1934 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:18318
[junit4:junit4]   2> 590 T1934 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:18318
[junit4:junit4]   2> 591 T1936 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c978df2140002 with negotiated timeout 20000 for client /140.211.11.196:18318
[junit4:junit4]   2> 591 T1951 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:18314, sessionid = 0x13c978df2140002, negotiated timeout = 20000
[junit4:junit4]   2> 592 T1952 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2c45fe62 name:ZooKeeperConnection Watcher:127.0.0.1:18314 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 592 T1932 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 593 T1937 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c978df2140002
[junit4:junit4]   2> 594 T1952 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 594 T1934 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:18318 which had sessionid 0x13c978df2140002
[junit4:junit4]   2> 594 T1932 oaz.ZooKeeper.close Session: 0x13c978df2140002 closed
[junit4:junit4]   2> 595 T1932 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 607 T1932 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:18314/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@448e2986
[junit4:junit4]   2> 608 T1932 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 608 T1953 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:18314. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 609 T1953 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:18314, initiating session
[junit4:junit4]   2> 609 T1934 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:18319
[junit4:junit4]   2> 609 T1934 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:18319
[junit4:junit4]   2> 611 T1936 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c978df2140003 with negotiated timeout 20000 for client /140.211.11.196:18319
[junit4:junit4]   2> 611 T1953 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:18314, sessionid = 0x13c978df2140003, negotiated timeout = 20000
[junit4:junit4]   2> 611 T1954 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@448e2986 name:ZooKeeperConnection Watcher:127.0.0.1:18314/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 611 T1932 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 613 T1937 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c978df2140003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 621 T1932 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 623 T1932 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:18317_r_ziz%2Fa
[junit4:junit4]   2> 623 T1937 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c978df2140003 type:delete cxid:0x7 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:18317_r_ziz%2Fa Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:18317_r_ziz%2Fa
[junit4:junit4]   2> 624 T1932 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:18317_r_ziz%2Fa
[junit4:junit4]   2> 633 T1932 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 647 T1937 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c978df2140003 type:delete cxid:0x15 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 648 T1932 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 651 T1932 oasc.Overseer.start Overseer (id=89112728548147203-127.0.0.1:18317_r_ziz%2Fa-n_0000000000) starting
[junit4:junit4]   2> 651 T1937 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c978df2140003 type:create cxid:0x1a zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 659 T1937 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c978df2140003 type:create cxid:0x1b zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 660 T1937 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c978df2140003 type:create cxid:0x1c zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 661 T1937 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c978df2140003 type:create cxid:0x1d zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 662 T1956 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 663 T1932 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 665 T1932 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 672 T1957 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1359752327996/collection1
[junit4:junit4]   2> 672 T1955 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 673 T1957 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 673 T1957 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 674 T1957 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 675 T1957 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1359752327996/collection1/'
[junit4:junit4]   2> 676 T1957 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1359752327996/collection1/lib/README' to classloader
[junit4:junit4]   2> 677 T1957 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1359752327996/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 730 T1957 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 804 T1957 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 905 T1957 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 911 T1957 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 1584 T1957 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1597 T1957 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1601 T1957 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1615 T1957 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1620 T1957 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1624 T1957 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1625 T1957 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1626 T1957 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1359752327996/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1359752327693/control/data/
[junit4:junit4]   2> 1627 T1957 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6b8099d3
[junit4:junit4]   2> 1627 T1957 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 1628 T1957 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/control/data forceNew:false
[junit4:junit4]   2> 1629 T1957 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/control/data
[junit4:junit4]   2> 1629 T1957 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1359752327693/control/data/index/
[junit4:junit4]   2> 1630 T1957 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1359752327693/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1631 T1957 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/control/data/index forceNew:false
[junit4:junit4]   2> 1637 T1957 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1359752327693/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5debf240),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 1637 T1957 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 1638 T1957 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/control/data/index
[junit4:junit4]   2> 1638 T1957 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 1640 T1957 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1640 T1957 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 1641 T1957 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1641 T1957 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 1641 T1957 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1642 T1957 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1642 T1957 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1642 T1957 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1643 T1957 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1643 T1957 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 1644 T1957 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 1644 T1957 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1645 T1957 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 1645 T1957 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 1645 T1957 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 1646 T1957 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1646 T1957 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1646 T1957 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1647 T1957 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1647 T1957 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1648 T1957 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1648 T1957 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1648 T1957 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1649 T1957 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1649 T1957 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 1650 T1957 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 1664 T1957 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1669 T1957 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 1670 T1957 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 1673 T1957 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/control/data
[junit4:junit4]   2> 1674 T1957 oass.SolrIndexSearcher.<init> Opening Searcher@5a93b3aa main
[junit4:junit4]   2> 1675 T1957 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1675 T1957 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1676 T1957 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 1682 T1957 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 1727 T1958 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5a93b3aa main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1731 T1957 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 1731 T1957 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 1732 T1937 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c978df2140003 type:create cxid:0x50 zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 2177 T1955 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2177 T1955 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:18317_r_ziz%2Fa",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:18317/r_ziz/a"}
[junit4:junit4]   2> 2178 T1955 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4]   2> 2178 T1955 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 2178 T1937 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c978df2140003 type:create cxid:0x58 zxid:0x2c txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 2197 T1954 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 2746 T1957 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 2746 T1957 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:18317/r_ziz/a collection:control_collection shard:shard1
[junit4:junit4]   2> 2747 T1957 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4]   2> 2763 T1937 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c978df2140003 type:delete cxid:0x72 zxid:0x36 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4]   2> 2764 T1957 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 2765 T1937 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c978df2140003 type:create cxid:0x73 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2775 T1957 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 2775 T1957 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 2775 T1957 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:18317/r_ziz/a/collection1/
[junit4:junit4]   2> 2776 T1957 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 2776 T1957 oasc.SyncStrategy.syncToMe http://127.0.0.1:18317/r_ziz/a/collection1/ has no replicas
[junit4:junit4]   2> 2776 T1957 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:18317/r_ziz/a/collection1/
[junit4:junit4]   2> 2776 T1957 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4]   2> 2780 T1937 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c978df2140003 type:create cxid:0x7d zxid:0x3b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3701 T1955 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3725 T1954 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 3751 T1957 oasc.ZkController.register We are http://127.0.0.1:18317/r_ziz/a/collection1/ and leader is http://127.0.0.1:18317/r_ziz/a/collection1/
[junit4:junit4]   2> 3751 T1957 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:18317/r_ziz/a
[junit4:junit4]   2> 3751 T1957 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 3751 T1957 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 3752 T1957 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 3754 T1957 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3755 T1932 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 3756 T1932 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 3756 T1932 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 3772 T1932 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 3781 T1932 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:18314/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@71f7a0d0
[junit4:junit4]   2> 3782 T1932 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 3783 T1959 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:18314. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 3783 T1959 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:18314, initiating session
[junit4:junit4]   2> 3783 T1934 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:18324
[junit4:junit4]   2> 3784 T1934 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:18324
[junit4:junit4]   2> 3785 T1936 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c978df2140004 with negotiated timeout 10000 for client /140.211.11.196:18324
[junit4:junit4]   2> 3785 T1959 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:18314, sessionid = 0x13c978df2140004, negotiated timeout = 10000
[junit4:junit4]   2> 3785 T1960 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@71f7a0d0 name:ZooKeeperConnection Watcher:127.0.0.1:18314/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 3786 T1932 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 3787 T1932 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 3789 T1932 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 3990 T1932 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 3993 T1932 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:18328
[junit4:junit4]   2> 3994 T1932 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 3994 T1932 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 3995 T1932 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1359752331477
[junit4:junit4]   2> 3995 T1932 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1359752331477/solr.xml
[junit4:junit4]   2> 3995 T1932 oasc.CoreContainer.<init> New CoreContainer 79511924
[junit4:junit4]   2> 3996 T1932 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1359752331477/'
[junit4:junit4]   2> 3997 T1932 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1359752331477/'
[junit4:junit4]   2> 4050 T1932 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 4051 T1932 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 4051 T1932 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 4052 T1932 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 4052 T1932 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 4053 T1932 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 4053 T1932 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 4054 T1932 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 4054 T1932 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 4055 T1932 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 4074 T1932 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 4093 T1932 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:18314/solr
[junit4:junit4]   2> 4094 T1932 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 4095 T1932 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:18314 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@4c6923b5
[junit4:junit4]   2> 4096 T1932 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4096 T1970 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:18314. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4097 T1970 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:18314, initiating session
[junit4:junit4]   2> 4097 T1934 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:18329
[junit4:junit4]   2> 4098 T1934 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:18329
[junit4:junit4]   2> 4104 T1936 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c978df2140005 with negotiated timeout 20000 for client /140.211.11.196:18329
[junit4:junit4]   2> 4104 T1970 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:18314, sessionid = 0x13c978df2140005, negotiated timeout = 20000
[junit4:junit4]   2> 4104 T1971 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4c6923b5 name:ZooKeeperConnection Watcher:127.0.0.1:18314 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4104 T1932 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4106 T1937 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c978df2140005
[junit4:junit4]   2> 4151 T1971 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 4151 T1934 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:18329 which had sessionid 0x13c978df2140005
[junit4:junit4]   2> 4151 T1932 oaz.ZooKeeper.close Session: 0x13c978df2140005 closed
[junit4:junit4]   2> 4152 T1932 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 4164 T1932 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:18314/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@5841e204
[junit4:junit4]   2> 4165 T1932 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4165 T1972 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:18314. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4166 T1972 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:18314, initiating session
[junit4:junit4]   2> 4166 T1934 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:18330
[junit4:junit4]   2> 4166 T1934 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:18330
[junit4:junit4]   2> 4192 T1936 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c978df2140006 with negotiated timeout 20000 for client /140.211.11.196:18330
[junit4:junit4]   2> 4192 T1972 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:18314, sessionid = 0x13c978df2140006, negotiated timeout = 20000
[junit4:junit4]   2> 4193 T1973 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5841e204 name:ZooKeeperConnection Watcher:127.0.0.1:18314/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4193 T1932 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4194 T1937 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c978df2140006 type:create cxid:0x1 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4242 T1937 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c978df2140006 type:create cxid:0x2 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4266 T1932 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:18328_r_ziz%2Fa
[junit4:junit4]   2> 4267 T1937 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c978df2140006 type:delete cxid:0x4 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:18328_r_ziz%2Fa Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:18328_r_ziz%2Fa
[junit4:junit4]   2> 4312 T1932 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:18328_r_ziz%2Fa
[junit4:junit4]   2> 4346 T1954 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 4346 T1960 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 4347 T1954 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 4349 T1932 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 4357 T1974 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1359752331477/collection1
[junit4:junit4]   2> 4357 T1974 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 4358 T1974 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 4358 T1974 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 4360 T1974 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1359752331477/collection1/'
[junit4:junit4]   2> 4361 T1974 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1359752331477/collection1/lib/README' to classloader
[junit4:junit4]   2> 4361 T1974 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1359752331477/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 4416 T1974 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 4489 T1974 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 4590 T1974 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 4597 T1974 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 5230 T1974 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 5236 T1955 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 5237 T1955 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:18317_r_ziz%2Fa",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:18317/r_ziz/a"}
[junit4:junit4]   2> 5238 T1974 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 5241 T1974 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 5268 T1960 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5268 T1973 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5268 T1954 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5276 T1974 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5279 T1974 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5283 T1974 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5284 T1974 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5285 T1974 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1359752331477/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty1/
[junit4:junit4]   2> 5285 T1974 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6b8099d3
[junit4:junit4]   2> 5285 T1974 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 5286 T1974 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty1 forceNew:false
[junit4:junit4]   2> 5287 T1974 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty1
[junit4:junit4]   2> 5287 T1974 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty1/index/
[junit4:junit4]   2> 5287 T1974 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 5288 T1974 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty1/index forceNew:false
[junit4:junit4]   2> 5304 T1974 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5885cf0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 5305 T1974 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 5306 T1974 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty1/index
[junit4:junit4]   2> 5306 T1974 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 5307 T1974 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 5308 T1974 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 5308 T1974 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 5308 T1974 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 5308 T1974 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 5309 T1974 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 5309 T1974 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 5309 T1974 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 5310 T1974 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 5310 T1974 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 5310 T1974 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 5311 T1974 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 5311 T1974 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 5311 T1974 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 5311 T1974 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 5312 T1974 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 5312 T1974 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5313 T1974 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5313 T1974 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5313 T1974 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5314 T1974 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5314 T1974 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5314 T1974 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5315 T1974 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5315 T1974 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 5315 T1974 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 5324 T1974 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 5327 T1974 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 5328 T1974 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 5330 T1974 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty1
[junit4:junit4]   2> 5331 T1974 oass.SolrIndexSearcher.<init> Opening Searcher@5c0609c3 main
[junit4:junit4]   2> 5332 T1974 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 5332 T1974 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 5333 T1974 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 5344 T1974 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 5404 T1975 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5c0609c3 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 5407 T1974 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 5407 T1974 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 6773 T1955 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6773 T1955 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:18328_r_ziz%2Fa",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:18328/r_ziz/a"}
[junit4:junit4]   2> 6774 T1955 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 1
[junit4:junit4]   2> 6774 T1955 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 6786 T1954 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6786 T1960 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6786 T1973 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7411 T1974 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 7411 T1974 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:18328/r_ziz/a collection:collection1 shard:shard1
[junit4:junit4]   2> 7412 T1974 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 7427 T1937 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c978df2140006 type:delete cxid:0x45 zxid:0x5b txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 7428 T1974 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 7429 T1937 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c978df2140006 type:create cxid:0x46 zxid:0x5c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7434 T1974 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 7434 T1974 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 7434 T1974 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:18328/r_ziz/a/collection1/
[junit4:junit4]   2> 7434 T1974 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 7435 T1974 oasc.SyncStrategy.syncToMe http://127.0.0.1:18328/r_ziz/a/collection1/ has no replicas
[junit4:junit4]   2> 7435 T1974 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:18328/r_ziz/a/collection1/
[junit4:junit4]   2> 7435 T1974 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 7453 T1937 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c978df2140006 type:create cxid:0x50 zxid:0x60 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8290 T1955 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8312 T1954 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8312 T1973 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8312 T1960 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8322 T1974 oasc.ZkController.register We are http://127.0.0.1:18328/r_ziz/a/collection1/ and leader is http://127.0.0.1:18328/r_ziz/a/collection1/
[junit4:junit4]   2> 8322 T1974 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:18328/r_ziz/a
[junit4:junit4]   2> 8322 T1974 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 8322 T1974 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 8323 T1974 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 8325 T1974 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8326 T1932 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 8327 T1932 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 8327 T1932 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 8512 T1932 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 8515 T1932 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:18335
[junit4:junit4]   2> 8516 T1932 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 8517 T1932 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 8517 T1932 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1359752336029
[junit4:junit4]   2> 8518 T1932 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1359752336029/solr.xml
[junit4:junit4]   2> 8518 T1932 oasc.CoreContainer.<init> New CoreContainer 1852995571
[junit4:junit4]   2> 8519 T1932 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1359752336029/'
[junit4:junit4]   2> 8519 T1932 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1359752336029/'
[junit4:junit4]   2> 8574 T1932 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 8575 T1932 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 8575 T1932 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 8576 T1932 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 8576 T1932 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 8577 T1932 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 8577 T1932 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 8578 T1932 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 8578 T1932 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 8578 T1932 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 8598 T1932 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 8619 T1932 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:18314/solr
[junit4:junit4]   2> 8619 T1932 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 8620 T1932 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:18314 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@7a1c53a3
[junit4:junit4]   2> 8621 T1932 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 8621 T1985 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:18314. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 8622 T1985 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:18314, initiating session
[junit4:junit4]   2> 8622 T1934 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:18336
[junit4:junit4]   2> 8623 T1934 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:18336
[junit4:junit4]   2> 8625 T1936 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c978df2140007 with negotiated timeout 20000 for client /140.211.11.196:18336
[junit4:junit4]   2> 8625 T1985 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:18314, sessionid = 0x13c978df2140007, negotiated timeout = 20000
[junit4:junit4]   2> 8625 T1986 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7a1c53a3 name:ZooKeeperConnection Watcher:127.0.0.1:18314 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 8625 T1932 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 8627 T1937 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c978df2140007
[junit4:junit4]   2> 8638 T1986 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 8638 T1934 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:18336 which had sessionid 0x13c978df2140007
[junit4:junit4]   2> 8638 T1932 oaz.ZooKeeper.close Session: 0x13c978df2140007 closed
[junit4:junit4]   2> 8639 T1932 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 8649 T1932 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:18314/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@b0e72ae
[junit4:junit4]   2> 8650 T1932 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 8651 T1987 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:18314. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 8651 T1987 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:18314, initiating session
[junit4:junit4]   2> 8652 T1934 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:18337
[junit4:junit4]   2> 8652 T1934 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:18337
[junit4:junit4]   2> 8653 T1936 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c978df2140008 with negotiated timeout 20000 for client /140.211.11.196:18337
[junit4:junit4]   2> 8654 T1987 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:18314, sessionid = 0x13c978df2140008, negotiated timeout = 20000
[junit4:junit4]   2> 8654 T1988 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b0e72ae name:ZooKeeperConnection Watcher:127.0.0.1:18314/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 8654 T1932 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 8655 T1937 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c978df2140008 type:create cxid:0x1 zxid:0x6d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8657 T1937 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c978df2140008 type:create cxid:0x2 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8664 T1932 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:18335_r_ziz%2Fa
[junit4:junit4]   2> 8665 T1937 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c978df2140008 type:delete cxid:0x4 zxid:0x6f txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:18335_r_ziz%2Fa Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:18335_r_ziz%2Fa
[junit4:junit4]   2> 8666 T1932 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:18335_r_ziz%2Fa
[junit4:junit4]   2> 8669 T1954 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 8669 T1973 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 8669 T1954 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8669 T1973 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8669 T1960 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 8670 T1960 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8677 T1932 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 8685 T1989 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1359752336029/collection1
[junit4:junit4]   2> 8685 T1989 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 8686 T1989 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 8686 T1989 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 8687 T1989 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1359752336029/collection1/'
[junit4:junit4]   2> 8688 T1989 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1359752336029/collection1/lib/README' to classloader
[junit4:junit4]   2> 8689 T1989 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1359752336029/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 8741 T1989 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 8817 T1989 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 8918 T1989 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 8924 T1989 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 9543 T1989 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 9556 T1989 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 9560 T1989 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 9574 T1989 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 9579 T1989 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 9583 T1989 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 9585 T1989 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 9586 T1989 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1359752336029/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty2/
[junit4:junit4]   2> 9586 T1989 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6b8099d3
[junit4:junit4]   2> 9587 T1989 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 9587 T1989 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty2 forceNew:false
[junit4:junit4]   2> 9588 T1989 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty2
[junit4:junit4]   2> 9589 T1989 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty2/index/
[junit4:junit4]   2> 9589 T1989 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 9590 T1989 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty2/index forceNew:false
[junit4:junit4]   2> 9595 T1989 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1fcb41b),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 9596 T1989 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 9596 T1989 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty2/index
[junit4:junit4]   2> 9597 T1989 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 9599 T1989 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 9599 T1989 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 9599 T1989 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 9600 T1989 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 9600 T1989 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 9600 T1989 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 9601 T1989 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 9601 T1989 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 9602 T1989 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 9602 T1989 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 9603 T1989 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 9604 T1989 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 9604 T1989 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 9604 T1989 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 9605 T1989 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 9605 T1989 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 9606 T1989 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9606 T1989 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9607 T1989 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9607 T1989 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9608 T1989 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9608 T1989 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9609 T1989 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9609 T1989 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9610 T1989 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 9610 T1989 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 9622 T1989 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 9626 T1989 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 9627 T1989 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 9630 T1989 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty2
[junit4:junit4]   2> 9631 T1989 oass.SolrIndexSearcher.<init> Opening Searcher@6d56aab2 main
[junit4:junit4]   2> 9632 T1989 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 9632 T1989 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 9633 T1989 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 9639 T1989 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 9693 T1990 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6d56aab2 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 9698 T1989 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 9698 T1989 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 9818 T1955 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9819 T1955 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:18328_r_ziz%2Fa",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:18328/r_ziz/a"}
[junit4:junit4]   2> 9823 T1955 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:18335_r_ziz%2Fa",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:18335/r_ziz/a"}
[junit4:junit4]   2> 9823 T1955 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4]   2> 9824 T1955 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 9836 T1954 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9836 T1973 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9836 T1960 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9836 T1988 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10701 T1989 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 10701 T1989 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:18335/r_ziz/a collection:collection1 shard:shard1
[junit4:junit4]   2> 10705 T1989 oasc.ZkController.register We are http://127.0.0.1:18335/r_ziz/a/collection1/ and leader is http://127.0.0.1:18328/r_ziz/a/collection1/
[junit4:junit4]   2> 10705 T1989 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:18335/r_ziz/a
[junit4:junit4]   2> 10705 T1989 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 10705 T1989 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C149 name=collection1 org.apache.solr.core.SolrCore@10a2c2f4 url=http://127.0.0.1:18335/r_ziz/a/collection1 node=127.0.0.1:18335_r_ziz%2Fa C149_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:18335_r_ziz%2Fa, base_url=http://127.0.0.1:18335/r_ziz/a}
[junit4:junit4]   2> 10706 T1991 C149 P18335 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 10706 T1989 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 10707 T1991 C149 P18335 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 10707 T1991 C149 P18335 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 10707 T1991 C149 P18335 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 10708 T1932 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 10709 T1991 C149 P18335 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 10709 T1932 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 10709 T1932 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 10954 T1932 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 10967 T1932 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:18352
[junit4:junit4]   2> 10968 T1932 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 10969 T1932 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 10969 T1932 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1359752338422
[junit4:junit4]   2> 10970 T1932 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1359752338422/solr.xml
[junit4:junit4]   2> 10970 T1932 oasc.CoreContainer.<init> New CoreContainer 832778518
[junit4:junit4]   2> 10971 T1932 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1359752338422/'
[junit4:junit4]   2> 10972 T1932 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1359752338422/'
[junit4:junit4]   2> 11080 T1932 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 11081 T1932 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 11082 T1932 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 11083 T1932 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 11083 T1932 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 11084 T1932 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 11085 T1932 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 11085 T1932 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 11086 T1932 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 11087 T1932 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 11126 T1932 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 11157 T1932 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:18314/solr
[junit4:junit4]   2> 11158 T1932 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 11158 T1932 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:18314 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@d56aafa
[junit4:junit4]   2> 11159 T1932 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 11160 T2001 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:18314. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 11161 T2001 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:18314, initiating session
[junit4:junit4]   2> 11161 T1934 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:18353
[junit4:junit4]   2> 11162 T1934 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:18353
[junit4:junit4]   2> 11164 T1936 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c978df2140009 with negotiated timeout 20000 for client /140.211.11.196:18353
[junit4:junit4]   2> 11164 T2001 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:18314, sessionid = 0x13c978df2140009, negotiated timeout = 20000
[junit4:junit4]   2> 11164 T2002 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d56aafa name:ZooKeeperConnection Watcher:127.0.0.1:18314 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 11165 T1932 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 11166 T1937 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c978df2140009
[junit4:junit4]   2> 11167 T2002 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 11167 T1934 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:18353 which had sessionid 0x13c978df2140009
[junit4:junit4]   2> 11167 T1932 oaz.ZooKeeper.close Session: 0x13c978df2140009 closed
[junit4:junit4]   2> 11168 T1932 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 11181 T1932 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:18314/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@f528d36
[junit4:junit4]   2> 11182 T1932 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 11182 T2003 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:18314. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 11183 T2003 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:18314, initiating session
[junit4:junit4]   2> 11183 T1934 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:18354
[junit4:junit4]   2> 11183 T1934 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:18354
[junit4:junit4]   2> 11186 T1936 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c978df214000a with negotiated timeout 20000 for client /140.211.11.196:18354
[junit4:junit4]   2> 11186 T2003 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:18314, sessionid = 0x13c978df214000a, negotiated timeout = 20000
[junit4:junit4]   2> 11187 T2004 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f528d36 name:ZooKeeperConnection Watcher:127.0.0.1:18314/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 11187 T1932 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 11189 T1937 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c978df214000a type:create cxid:0x1 zxid:0x7f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 11190 T1937 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c978df214000a type:create cxid:0x2 zxid:0x80 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 11192 T1932 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:18352_r_ziz%2Fa
[junit4:junit4]   2> 11193 T1937 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c978df214000a type:delete cxid:0x4 zxid:0x81 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:18352_r_ziz%2Fa Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:18352_r_ziz%2Fa
[junit4:junit4]   2> 11200 T1932 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:18352_r_ziz%2Fa
[junit4:junit4]   2> 11203 T1988 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 11204 T1960 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 11204 T1988 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11204 T1954 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 11204 T1960 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11204 T1973 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 11204 T1954 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11205 T1973 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11208 T1932 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 11219 T2005 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1359752338422/collection1
[junit4:junit4]   2> 11220 T2005 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 11220 T2005 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 11221 T2005 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 11222 T2005 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1359752338422/collection1/'
[junit4:junit4]   2> 11223 T2005 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1359752338422/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 11224 T2005 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1359752338422/collection1/lib/README' to classloader
[junit4:junit4]   2> 11283 T2005 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 11349 T1955 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11351 T1955 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:18335_r_ziz%2Fa",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:18335/r_ziz/a"}
[junit4:junit4]   2> 11360 T2004 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11360 T1954 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11360 T1960 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11360 T1973 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11360 T1988 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11404 T2005 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 11505 T2005 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 11512 T2005 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 12128 T2005 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 12136 T2005 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 12139 T2005 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 12149 T2005 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 12153 T2005 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 12156 T2005 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 12157 T2005 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 12158 T2005 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1359752338422/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty3/
[junit4:junit4]   2> 12158 T2005 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6b8099d3
[junit4:junit4]   2> 12158 T2005 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 12159 T2005 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty3 forceNew:false
[junit4:junit4]   2> 12159 T2005 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty3
[junit4:junit4]   2> 12160 T2005 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty3/index/
[junit4:junit4]   2> 12160 T2005 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 12161 T2005 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty3/index forceNew:false
[junit4:junit4]   2> 12165 T2005 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6d0428aa),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 12165 T2005 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 12165 T2005 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty3/index
[junit4:junit4]   2> 12166 T2005 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 12167 T2005 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 12167 T2005 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 12168 T2005 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 12168 T2005 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 12168 T2005 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 12168 T2005 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 12169 T2005 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 12169 T2005 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 12169 T2005 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 12170 T2005 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 12170 T2005 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 12170 T2005 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 12171 T2005 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 12171 T2005 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 12171 T2005 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 12172 T2005 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 12172 T2005 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12172 T2005 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12173 T2005 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12173 T2005 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12173 T2005 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12174 T2005 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12174 T2005 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12174 T2005 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 12175 T2005 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 12175 T2005 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 12184 T2005 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 12187 T2005 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 12188 T2005 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 12189 T2005 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty3
[junit4:junit4]   2> 12190 T2005 oass.SolrIndexSearcher.<init> Opening Searcher@4c133b2c main
[junit4:junit4]   2> 12191 T2005 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 12191 T2005 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 12192 T2005 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 12196 T2005 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 12240 T2006 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4c133b2c main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 12244 T2005 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 12244 T2005 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 12871 T1955 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12872 T1955 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:18352_r_ziz%2Fa",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:18352/r_ziz/a"}
[junit4:junit4]   2> 12872 T1955 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4]   2> 12872 T1955 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 12884 T1988 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12884 T2004 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12884 T1954 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12884 T1973 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12884 T1960 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13246 T2005 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 13246 T2005 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:18352/r_ziz/a collection:collection1 shard:shard1
[junit4:junit4]   2> 13249 T2005 oasc.ZkController.register We are http://127.0.0.1:18352/r_ziz/a/collection1/ and leader is http://127.0.0.1:18328/r_ziz/a/collection1/
[junit4:junit4]   2> 13249 T2005 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:18352/r_ziz/a
[junit4:junit4]   2> 13249 T2005 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 13250 T2005 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C150 name=collection1 org.apache.solr.core.SolrCore@1d2cef85 url=http://127.0.0.1:18352/r_ziz/a/collection1 node=127.0.0.1:18352_r_ziz%2Fa C150_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:18352_r_ziz%2Fa, base_url=http://127.0.0.1:18352/r_ziz/a}
[junit4:junit4]   2> 13250 T2007 C150 P18352 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 13250 T2005 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 13251 T2007 C150 P18352 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 13251 T2007 C150 P18352 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 13251 T2007 C150 P18352 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 13252 T1932 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 13252 T1932 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 13253 T2007 C150 P18352 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 13253 T1932 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 13442 T1932 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 13445 T1932 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:18357
[junit4:junit4]   2> 13446 T1932 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 13446 T1932 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 13447 T1932 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1359752340962
[junit4:junit4]   2> 13447 T1932 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1359752340962/solr.xml
[junit4:junit4]   2> 13448 T1932 oasc.CoreContainer.<init> New CoreContainer 1634914841
[junit4:junit4]   2> 13448 T1932 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1359752340962/'
[junit4:junit4]   2> 13449 T1932 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1359752340962/'
[junit4:junit4]   2> 13503 T1932 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 13503 T1932 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 13504 T1932 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 13504 T1932 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 13505 T1932 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 13505 T1932 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 13506 T1932 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 13506 T1932 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 13507 T1932 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 13507 T1932 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 13527 T1932 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 13548 T1932 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:18314/solr
[junit4:junit4]   2> 13548 T1932 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 13549 T1932 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:18314 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@7154583f
[junit4:junit4]   2> 13550 T1932 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 13550 T2017 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:18314. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 13551 T2017 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:18314, initiating session
[junit4:junit4]   2> 13551 T1934 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:18358
[junit4:junit4]   2> 13552 T1934 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:18358
[junit4:junit4]   2> 13554 T1936 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c978df214000b with negotiated timeout 20000 for client /140.211.11.196:18358
[junit4:junit4]   2> 13554 T2017 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:18314, sessionid = 0x13c978df214000b, negotiated timeout = 20000
[junit4:junit4]   2> 13554 T2018 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7154583f name:ZooKeeperConnection Watcher:127.0.0.1:18314 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 13554 T1932 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 13556 T1937 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c978df214000b
[junit4:junit4]   2> 13562 T2018 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 13562 T1934 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:18358 which had sessionid 0x13c978df214000b
[junit4:junit4]   2> 13562 T1932 oaz.ZooKeeper.close Session: 0x13c978df214000b closed
[junit4:junit4]   2> 13563 T1932 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 13573 T1932 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:18314/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@6b63f13c
[junit4:junit4]   2> 13574 T1932 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 13574 T2019 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:18314. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 13575 T2019 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:18314, initiating session
[junit4:junit4]   2> 13575 T1934 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:18359
[junit4:junit4]   2> 13575 T1934 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:18359
[junit4:junit4]   2> 13576 T1936 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c978df214000c with negotiated timeout 20000 for client /140.211.11.196:18359
[junit4:junit4]   2> 13576 T2019 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:18314, sessionid = 0x13c978df214000c, negotiated timeout = 20000
[junit4:junit4]   2> 13577 T2020 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6b63f13c name:ZooKeeperConnection Watcher:127.0.0.1:18314/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 13577 T1932 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 13578 T1937 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c978df214000c type:create cxid:0x1 zxid:0x92 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 13584 T1937 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c978df214000c type:create cxid:0x2 zxid:0x93 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 13585 T1932 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:18357_r_ziz%2Fa
[junit4:junit4]   2> 13586 T1937 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c978df214000c type:delete cxid:0x4 zxid:0x94 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:18357_r_ziz%2Fa Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:18357_r_ziz%2Fa
[junit4:junit4]   2> 13592 T1932 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:18357_r_ziz%2Fa
[junit4:junit4]   2> 13601 T2004 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13602 T1988 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 13602 T1973 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 13602 T1954 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 13602 T1960 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 13602 T1988 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 13602 T1973 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 13602 T1954 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 13603 T1960 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 13603 T2004 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 13606 T1932 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 13614 T2021 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1359752340962/collection1
[junit4:junit4]   2> 13615 T2021 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 13616 T2021 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 13616 T2021 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 13617 T2021 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1359752340962/collection1/'
[junit4:junit4]   2> 13619 T2021 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty4-1359752340962/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 13620 T2021 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty4-1359752340962/collection1/lib/README' to classloader
[junit4:junit4]   2> 13678 T2021 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2>  C149_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:18335_r_ziz%2Fa, base_url=http://127.0.0.1:18335/r_ziz/a}
[junit4:junit4]   2> 13737 T1991 C149 P18335 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:18328/r_ziz/a/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 13737 T1991 C149 P18335 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:18335/r_ziz/a START replicas=[http://127.0.0.1:18328/r_ziz/a/collection1/] nUpdates=100
[junit4:junit4]   2> 13738 T1991 C149 P18335 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 13738 T1991 C149 P18335 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 13738 T1991 C149 P18335 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 13738 T1991 C149 P18335 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 13739 T1991 C149 P18335 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 13739 T1991 C149 P18335 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:18328/r_ziz/a/collection1/. core=collection1
[junit4:junit4]   2> 13739 T1991 C149 P18335 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C151 name=collection1 org.apache.solr.core.SolrCore@51345b3c url=http://127.0.0.1:18328/r_ziz/a/collection1 node=127.0.0.1:18328_r_ziz%2Fa C151_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:18328_r_ziz%2Fa, base_url=http://127.0.0.1:18328/r_ziz/a, leader=true}
[junit4:junit4]   2> 13749 T1965 C151 P18328 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 13753 T1966 C151 P18328 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 13755 T1966 C151 P18328 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty1
[junit4:junit4]   2> 13762 T1966 C151 P18328 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5885cf0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 13763 T1966 C151 P18328 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 13770 T1966 C151 P18328 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5885cf0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5885cf0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 13770 T1966 C151 P18328 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 13773 T1966 C151 P18328 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty1
[junit4:junit4]   2> 13773 T1966 C151 P18328 oass.SolrIndexSearcher.<init> Opening Searcher@c9d435e realtime
[junit4:junit4]   2> 13773 T1966 C151 P18328 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 13774 T2021 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 13774 T1966 C151 P18328 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 21
[junit4:junit4]   2> 13775 T1991 C149 P18335 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 13775 T1991 C149 P18335 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 13778 T1967 C151 P18328 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 13779 T1967 C151 P18328 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 13779 T1991 C149 P18335 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 13780 T1991 C149 P18335 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 13780 T1991 C149 P18335 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 13782 T1967 C151 P18328 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty1
[junit4:junit4]   2> 13784 T1967 C151 P18328 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty1/index
[junit4:junit4]   2> 13784 T1967 C151 P18328 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=2 
[junit4:junit4]   2> 13785 T1991 C149 P18335 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 13787 T1991 C149 P18335 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty2/index.20130201125901473 forceNew:false
[junit4:junit4]   2> 13787 T1991 C149 P18335 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty2
[junit4:junit4]   2> 13789 T1991 C149 P18335 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty2/index.20130201125901473 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b0d60dd) fullCopy=true
[junit4:junit4]   2> 13793 T1967 C151 P18328 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> 13795 T1991 C149 P18335 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty2/index
[junit4:junit4]   2> 13795 T1991 C149 P18335 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 13796 T1991 C149 P18335 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130201125901473
[junit4:junit4]   2> 13799 T1991 C149 P18335 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty2
[junit4:junit4]   2> 13800 T1991 C149 P18335 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty2
[junit4:junit4]   2> 13800 T1991 C149 P18335 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty2/index/ new=./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty2/index.20130201125901473
[junit4:junit4]   2> 13809 T1991 C149 P18335 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty2/index.20130201125901473 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b0d60dd),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 13810 T1991 C149 P18335 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 13818 T1991 C149 P18335 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty2
[junit4:junit4]   2> 13819 T1991 C149 P18335 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 13819 T1991 C149 P18335 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 13820 T1991 C149 P18335 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 13821 T1991 C149 P18335 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty2/index.20130201125901473
[junit4:junit4]   2> 13823 T1991 C149 P18335 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty2
[junit4:junit4]   2> 13827 T1991 C149 P18335 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty2/index.20130201125901473 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b0d60dd),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 13828 T1991 C149 P18335 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 13828 T1991 C149 P18335 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 13829 T1991 C149 P18335 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty2
[junit4:junit4]   2> 13829 T1991 C149 P18335 oass.SolrIndexSearcher.<init> Opening Searcher@645edb48 main
[junit4:junit4]   2> 13831 T1990 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@645edb48 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 13831 T1990 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty2/index
[junit4:junit4]   2> 13832 T1991 C149 P18335 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 13869 T1991 C149 P18335 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty2/index.20130201125901473 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b0d60dd),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty2/index.20130201125901473 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b0d60dd),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 13869 T1991 C149 P18335 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 13870 T1991 C149 P18335 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty2
[junit4:junit4]   2> 13871 T1991 C149 P18335 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 13871 T1991 C149 P18335 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty2/index.20130201125901473
[junit4:junit4]   2> 13871 T1991 C149 P18335 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty2/index
[junit4:junit4]   2> 13871 T1991 C149 P18335 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 13872 T1991 C149 P18335 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 13872 T1991 C149 P18335 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 13872 T1991 C149 P18335 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 13875 T2021 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 13885 T1991 C149 P18335 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 13895 T2021 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 14389 T1955 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14390 T1955 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:18352_r_ziz%2Fa",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:18352/r_ziz/a"}
[junit4:junit4]   2> 14399 T1955 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:18335_r_ziz%2Fa",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:18335/r_ziz/a"}
[junit4:junit4]   2> 14412 T2020 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14412 T1960 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14412 T1988 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14412 T1973 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14412 T2004 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14412 T1954 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14779 T2021 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 14803 T2021 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 14810 T2021 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 14828 T2021 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 14837 T2021 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 14845 T2021 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 14846 T2021 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 14848 T2021 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1359752340962/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty4/
[junit4:junit4]   2> 14848 T2021 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6b8099d3
[junit4:junit4]   2> 14849 T2021 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 14850 T2021 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty4 forceNew:false
[junit4:junit4]   2> 14851 T2021 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty4
[junit4:junit4]   2> 14851 T2021 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty4/index/
[junit4:junit4]   2> 14852 T2021 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 14854 T2021 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty4/index forceNew:false
[junit4:junit4]   2> 14863 T2021 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@aa32589),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 14863 T2021 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 14864 T2021 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty4/index
[junit4:junit4]   2> 14864 T2021 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 14866 T2021 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 14867 T2021 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 14867 T2021 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 14867 T2021 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 14868 T2021 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 14868 T2021 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 14869 T2021 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 14870 T2021 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 14871 T2021 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 14872 T2021 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 14872 T2021 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 14873 T2021 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 14874 T2021 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 14874 T2021 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 14875 T2021 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 14876 T2021 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 14876 T2021 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14877 T2021 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14878 T2021 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14879 T2021 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14879 T2021 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14880 T2021 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14881 T2021 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14882 T2021 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14883 T2021 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 14884 T2021 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 14914 T2021 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 14924 T2021 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 14926 T2021 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 14930 T2021 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty4
[junit4:junit4]   2> 14934 T2021 oass.SolrIndexSearcher.<init> Opening Searcher@6e822855 main
[junit4:junit4]   2> 14936 T2021 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 14936 T2021 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 14937 T2021 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 15008 T2021 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 15060 T2024 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6e822855 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 15065 T2021 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 15066 T2021 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 15921 T1955 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 15922 T1955 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:18357_r_ziz%2Fa",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:18357/r_ziz/a"}
[junit4:junit4]   2> 15923 T1955 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4]   2> 15923 T1955 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 15935 T2004 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15935 T1960 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15935 T2020 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15935 T1988 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15935 T1954 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15935 T1973 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16069 T2021 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 16069 T2021 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:18357/r_ziz/a collection:collection1 shard:shard1
[junit4:junit4]   2> 16072 T2021 oasc.ZkController.register We are http://127.0.0.1:18357/r_ziz/a/collection1/ and leader is http://127.0.0.1:18328/r_ziz/a/collection1/
[junit4:junit4]   2> 16073 T2021 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:18357/r_ziz/a
[junit4:junit4]   2> 16073 T2021 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 16073 T2021 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C152 name=collection1 org.apache.solr.core.SolrCore@62ed8299 url=http://127.0.0.1:18357/r_ziz/a/collection1 node=127.0.0.1:18357_r_ziz%2Fa C152_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:18357_r_ziz%2Fa, base_url=http://127.0.0.1:18357/r_ziz/a}
[junit4:junit4]   2> 16074 T2025 C152 P18357 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 16074 T2025 C152 P18357 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 16074 T2021 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16075 T2025 C152 P18357 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 16075 T2025 C152 P18357 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 16075 T1932 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 16076 T1932 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 16076 T2025 C152 P18357 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 16077 T1932 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 16092 T1932 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16093 T1932 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4]   2> 16093 T1932 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4]   2> 16094 T1932 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17096 T1932 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C150_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:18352_r_ziz%2Fa, base_url=http://127.0.0.1:18352/r_ziz/a}
[junit4:junit4]   2> 17278 T2007 C150 P18352 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:18328/r_ziz/a/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 17278 T2007 C150 P18352 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:18352/r_ziz/a START replicas=[http://127.0.0.1:18328/r_ziz/a/collection1/] nUpdates=100
[junit4:junit4]   2> 17278 T2007 C150 P18352 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 17279 T2007 C150 P18352 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 17279 T2007 C150 P18352 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 17279 T2007 C150 P18352 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 17279 T2007 C150 P18352 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 17279 T2007 C150 P18352 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:18328/r_ziz/a/collection1/. core=collection1
[junit4:junit4]   2> 17280 T2007 C150 P18352 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 17281 T1965 C151 P18328 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 17293 T1969 C151 P18328 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 17297 T1969 C151 P18328 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5885cf0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5885cf0),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 17298 T1969 C151 P18328 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 17298 T1969 C151 P18328 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty1
[junit4:junit4]   2> 17299 T1969 C151 P18328 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 17299 T1969 C151 P18328 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 7
[junit4:junit4]   2> 17300 T2007 C150 P18352 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 17300 T2007 C150 P18352 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 17303 T1967 C151 P18328 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 17304 T2007 C150 P18352 oash.SnapPuller.fetchLatestIndex Master's generation: 3
[junit4:junit4]   2> 17304 T2007 C150 P18352 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 17304 T2007 C150 P18352 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 17307 T1967 C151 P18328 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty1
[junit4:junit4]   2> 17308 T1967 C151 P18328 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty1/index
[junit4:junit4]   2> 17308 T1967 C151 P18328 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=3&version=2} status=0 QTime=1 
[junit4:junit4]   2> 17309 T2007 C150 P18352 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 17310 T2007 C150 P18352 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty3/index.20130201125904997 forceNew:false
[junit4:junit4]   2> 17311 T2007 C150 P18352 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty3
[junit4:junit4]   2> 17311 T2007 C150 P18352 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty3/index.20130201125904997 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2ef3fcbf) fullCopy=true
[junit4:junit4]   2> 17315 T1967 C151 P18328 REQ /replication {file=segments_3&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=3} status=0 QTime=0 
[junit4:junit4]   2> 17316 T2007 C150 P18352 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty3/index
[junit4:junit4]   2> 17317 T2007 C150 P18352 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 17317 T2007 C150 P18352 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130201125904997
[junit4:junit4]   2> 17318 T2007 C150 P18352 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty3
[junit4:junit4]   2> 17319 T2007 C150 P18352 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty3
[junit4:junit4]   2> 17319 T2007 C150 P18352 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty3/index/ new=./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty3/index.20130201125904997
[junit4:junit4]   2> 17322 T2007 C150 P18352 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty3/index.20130201125904997 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2ef3fcbf),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 17323 T2007 C150 P18352 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 17324 T2007 C150 P18352 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty3
[junit4:junit4]   2> 17324 T2007 C150 P18352 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 17324 T2007 C150 P18352 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 17325 T2007 C150 P18352 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 17325 T2007 C150 P18352 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty3/index.20130201125904997
[junit4:junit4]   2> 17326 T2007 C150 P18352 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty3
[junit4:junit4]   2> 17329 T2007 C150 P18352 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty3/index.20130201125904997 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2ef3fcbf),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 17329 T2007 C150 P18352 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 17330 T2007 C150 P18352 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 17330 T2007 C150 P18352 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty3
[junit4:junit4]   2> 17330 T2007 C150 P18352 oass.SolrIndexSearcher.<init> Opening Searcher@7725bcfa main
[junit4:junit4]   2> 17332 T2006 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7725bcfa main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4]   2> 17333 T2006 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty3/index
[junit4:junit4]   2> 17333 T2007 C150 P18352 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 17335 T2007 C150 P18352 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty3/index.20130201125904997 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2ef3fcbf),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty3/index.20130201125904997 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2ef3fcbf),segFN=segments_4,generation=4,filenames=[segments_4]
[junit4:junit4]   2> 17335 T2007 C150 P18352 oasc.SolrDeletionPolicy.updateCommits newest commit = 4[segments_4]
[junit4:junit4]   2> 17336 T2007 C150 P18352 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty3
[junit4:junit4]   2> 17336 T2007 C150 P18352 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 17337 T2007 C150 P18352 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty3/index.20130201125904997
[junit4:junit4]   2> 17337 T2007 C150 P18352 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty3/index
[junit4:junit4]   2> 17337 T2007 C150 P18352 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 17337 T2007 C150 P18352 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 17337 T2007 C150 P18352 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 17338 T2007 C150 P18352 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 17345 T2007 C150 P18352 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 17440 T1955 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17441 T1955 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:18357_r_ziz%2Fa",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:18357/r_ziz/a"}
[junit4:junit4]   2> 17445 T1955 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:18352_r_ziz%2Fa",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:18352/r_ziz/a"}
[junit4:junit4]   2> 17457 T1954 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17457 T1988 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17457 T1973 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17457 T2020 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17457 T1960 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17457 T2004 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 18098 T1932 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19100 T1932 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C152_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:18357_r_ziz%2Fa, base_url=http://127.0.0.1:18357/r_ziz/a}
[junit4:junit4]   2> 20096 T2025 C152 P18357 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:18328/r_ziz/a/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 20096 T2025 C152 P18357 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:18357/r_ziz/a START replicas=[http://127.0.0.1:18328/r_ziz/a/collection1/] nUpdates=100
[junit4:junit4]   2> 20096 T2025 C152 P18357 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 20097 T2025 C152 P18357 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 20097 T2025 C152 P18357 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 20097 T2025 C152 P18357 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 20097 T2025 C152 P18357 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 20097 T2025 C152 P18357 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:18328/r_ziz/a/collection1/. core=collection1
[junit4:junit4]   2> 20097 T2025 C152 P18357 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 20099 T1965 C151 P18328 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 20103 T1932 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 20108 T1963 C151 P18328 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 20111 T1963 C151 P18328 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=3
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5885cf0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5885cf0),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5885cf0),segFN=segments_4,generation=4,filenames=[segments_4]
[junit4:junit4]   2> 20111 T1963 C151 P18328 oasc.SolrDeletionPolicy.updateCommits newest commit = 4[segments_4]
[junit4:junit4]   2> 20112 T1963 C151 P18328 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty1
[junit4:junit4]   2> 20112 T1963 C151 P18328 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 20113 T1963 C151 P18328 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 5
[junit4:junit4]   2> 20113 T2025 C152 P18357 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 20113 T2025 C152 P18357 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 20116 T1967 C151 P18328 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 20116 T2025 C152 P18357 oash.SnapPuller.fetchLatestIndex Master's generation: 4
[junit4:junit4]   2> 20117 T2025 C152 P18357 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 20117 T2025 C152 P18357 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 20119 T1967 C151 P18328 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty1
[junit4:junit4]   2> 20119 T1967 C151 P18328 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty1/index
[junit4:junit4]   2> 20120 T1967 C151 P18328 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=4&version=2} status=0 QTime=1 
[junit4:junit4]   2> 20120 T2025 C152 P18357 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 20121 T2025 C152 P18357 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty4/index.20130201125907808 forceNew:false
[junit4:junit4]   2> 20122 T2025 C152 P18357 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty4
[junit4:junit4]   2> 20122 T2025 C152 P18357 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty4/index.20130201125907808 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6bca1f4e) fullCopy=true
[junit4:junit4]   2> 20125 T1967 C151 P18328 REQ /replication {file=segments_4&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=4} status=0 QTime=0 
[junit4:junit4]   2> 20126 T2025 C152 P18357 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty4/index
[junit4:junit4]   2> 20126 T2025 C152 P18357 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 20127 T2025 C152 P18357 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130201125907808
[junit4:junit4]   2> 20128 T2025 C152 P18357 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty4
[junit4:junit4]   2> 20128 T2025 C152 P18357 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty4
[junit4:junit4]   2> 20129 T2025 C152 P18357 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty4/index/ new=./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty4/index.20130201125907808
[junit4:junit4]   2> 20132 T2025 C152 P18357 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty4/index.20130201125907808 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6bca1f4e),segFN=segments_4,generation=4,filenames=[segments_4]
[junit4:junit4]   2> 20132 T2025 C152 P18357 oasc.SolrDeletionPolicy.updateCommits newest commit = 4[segments_4]
[junit4:junit4]   2> 20133 T2025 C152 P18357 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty4
[junit4:junit4]   2> 20133 T2025 C152 P18357 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 20134 T2025 C152 P18357 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 20134 T2025 C152 P18357 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 20135 T2025 C152 P18357 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty4/index.20130201125907808
[junit4:junit4]   2> 20135 T2025 C152 P18357 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty4
[junit4:junit4]   2> 20138 T2025 C152 P18357 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty4/index.20130201125907808 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6bca1f4e),segFN=segments_4,generation=4,filenames=[segments_4]
[junit4:junit4]   2> 20138 T2025 C152 P18357 oasc.SolrDeletionPolicy.updateCommits newest commit = 4[segments_4]
[junit4:junit4]   2> 20139 T2025 C152 P18357 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 20139 T2025 C152 P18357 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1359752327693/jetty4
[junit4:junit4]   2> 20139 T2025 C152 P18357 oass.SolrIndexSearcher.<init> Open

[...truncated too long message...]

   > 	at org.junit.Assert.assertNotNull(Assert.java:526)
[junit4:junit4]    > 	at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:214)
[junit4:junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:794)
[junit4:junit4]    > 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4:junit4]    > 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[junit4:junit4]    > 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4:junit4]    > 	at java.lang.reflect.Method.invoke(Method.java:601)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4:junit4]    > 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]    > 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]    > 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 147136 T1932 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4]   2> NOTE: test params are: codec=Lucene42: {range_facet_l=PostingsFormat(name=SimpleText), text=PostingsFormat(name=NestedPulsing), _version_=PostingsFormat(name=SimpleText), rnd_b=MockVariableIntBlock(baseBlockSize=65), intDefault=PostingsFormat(name=SimpleText), timestamp=PostingsFormat(name=SimpleText), id=MockVariableIntBlock(baseBlockSize=65), a_t=PostingsFormat(name=SimpleText), range_facet_sl=PostingsFormat(name=NestedPulsing), range_facet_si=PostingsFormat(name=Lucene41WithOrds), other_tl1=PostingsFormat(name=SimpleText), multiDefault=PostingsFormat(name=Lucene41WithOrds), a_si=PostingsFormat(name=Lucene41WithOrds)}, sim=DefaultSimilarity, locale=pl_PL, timezone=Pacific/Pitcairn
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=2,free=153950936,total=451936256
[junit4:junit4]   2> NOTE: All tests run in this JVM: [TestBinaryResponseWriter, TestJoin, DirectUpdateHandlerOptimizeTest, TestLazyCores, DirectSolrConnectionTest, SpellCheckCollatorTest, SuggesterFSTTest, TestNumberUtils, SolrCoreCheckLockOnStartupTest, TestComponentsName, ZkControllerTest, AlternateDirectoryTest, TestQuerySenderNoQuery, TestLMDirichletSimilarityFactory, SimplePostToolTest, WordBreakSolrSpellCheckerTest, MoreLikeThisHandlerTest, TestFiltering, NumericFieldsTest, QueryElevationComponentTest, SuggesterWFSTTest, DateFieldTest, TestConfig, TestCoreContainer, TestRangeQuery, PreAnalyzedFieldTest, TestJmxMonitoredMap, DateMathParserTest, TestFastLRUCache, TestWordDelimiterFilterFactory, TestIndexingPerformance, FileBasedSpellCheckerTest, TestPostingsSolrHighlighter, TestXIncludeConfig, TestLuceneMatchVersion, SpellCheckComponentTest, TermsComponentTest, SortByFunctionTest, TestSuggestSpellingConverter, DefaultValueUpdateProcessorTest, CoreAdminHandlerTest, ChaosMonkeyNothingIsSafeTest, CSVRequestHandlerTest, SolrPluginUtilsTest, DistributedSpellCheckComponentTest, TestIndexSearcher, TestPerFieldSimilarity, ClusterStateUpdateTest, TestZkChroot, TestLFUCache, TestPHPSerializedResponseWriter, TestDocumentBuilder, TestPartialUpdateDeduplication, TestSolrIndexConfig, LeaderElectionIntegrationTest, TestSolr4Spatial, TestElisionMultitermQuery, MinimalSchemaTest, TestFuzzyAnalyzedSuggestions, LoggingHandlerTest, BasicDistributedZkTest, TestDistributedSearch, SoftAutoCommitTest, TestFaceting, DistributedTermsComponentTest, SolrCoreTest, IndexBasedSpellCheckerTest, DirectUpdateHandlerTest, TestMultiCoreConfBootstrap, TestSort, BasicFunctionalityTest, SignatureUpdateProcessorFactoryTest, SampleTest, SolrRequestParserTest, TestRemoteStreaming, CacheHeaderTest, QueryEqualityTest, SpatialFilterTest, TestCSVResponseWriter, MBeansHandlerTest, CurrencyFieldTest, TestPseudoReturnFields, IndexSchemaTest, BinaryUpdateRequestHandlerTest, DistanceFunctionTest, TestQueryTypes, JsonLoaderTest, DocumentAnalysisRequestHandlerTest, TestReversedWildcardFilterFactory, RequestHandlersTest, RequiredFieldsTest, CopyFieldTest, TestQueryUtils, SolrCmdDistributorTest, TestSearchPerf, NotRequiredUniqueKeyTest, UpdateRequestProcessorFactoryTest, TestWriterPerf, NoCacheHeaderTest, BadComponentTest, SOLR749Test, TestPropInjectDefaults, TestCollationField, OutputWriterTest, SpellPossibilityIteratorTest, TestPhraseSuggestions, TestIBSimilarityFactory, TestDefaultSimilarityFactory, PluginInfoTest, OpenExchangeRatesOrgProviderTest, FileUtilsTest, TestSolrXMLSerializer, DOMUtilTest, ChaosMonkeySafeLeaderTest, UUIDFieldTest, RAMDirectoryFactoryTest, SystemInfoHandlerTest, TestUtils, ZkNodePropsTest, CircularListTest, AnalysisAfterCoreReloadTest, ShardRoutingCustomTest, SyncSliceTest]
[junit4:junit4] Completed on J0 in 147.16s, 1 test, 1 failure <<< FAILURES!

[...truncated 34 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:346: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/common-build.xml:428: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:1176: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:841: There were test failures: 249 suites, 1038 tests, 1 failure, 23 ignored (5 assumptions)

Total time: 34 minutes 31 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-trunk-java7 - Build # 3698 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-java7/3698/

2 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([FBCDCC7974E9B069]:0)


FAILED:  org.apache.solr.cloud.BasicDistributedZkTest.testDistribSearch

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([FBCDCC7974E9B069]:0)




Build Log:
[...truncated 9501 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
[junit4:junit4]   2> 0 T1736 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /o_z/
[junit4:junit4]   2> 6 T1736 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZkTest-1359721446395
[junit4:junit4]   2> 7 T1736 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 8 T1737 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 11 T1737 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 11 T1737 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4]   2> 11 T1737 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4]   2> 12 T1737 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 13 T1737 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4]   2> 109 T1736 oasc.ZkTestServer.run start zk server on port:14566
[junit4:junit4]   2> 109 T1736 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:14566 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@576f4a80
[junit4:junit4]   2> 110 T1736 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 110 T1742 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:14566. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 111 T1742 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:14566, initiating session
[junit4:junit4]   2> 111 T1738 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:13910
[junit4:junit4]   2> 112 T1738 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:13910
[junit4:junit4]   2> 112 T1740 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 115 T1740 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c95b6bc020000 with negotiated timeout 10000 for client /140.211.11.196:13910
[junit4:junit4]   2> 115 T1742 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:14566, sessionid = 0x13c95b6bc020000, negotiated timeout = 10000
[junit4:junit4]   2> 116 T1743 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@576f4a80 name:ZooKeeperConnection Watcher:127.0.0.1:14566 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 116 T1736 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 116 T1736 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 119 T1741 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c95b6bc020000
[junit4:junit4]   2> 132 T1743 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 132 T1738 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:13910 which had sessionid 0x13c95b6bc020000
[junit4:junit4]   2> 132 T1736 oaz.ZooKeeper.close Session: 0x13c95b6bc020000 closed
[junit4:junit4]   2> 133 T1736 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:14566/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@173654d0
[junit4:junit4]   2> 134 T1736 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 134 T1744 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:14566. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 134 T1744 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:14566, initiating session
[junit4:junit4]   2> 135 T1738 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:24728
[junit4:junit4]   2> 135 T1738 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:24728
[junit4:junit4]   2> 136 T1740 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c95b6bc020001 with negotiated timeout 10000 for client /140.211.11.196:24728
[junit4:junit4]   2> 136 T1744 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:14566, sessionid = 0x13c95b6bc020001, negotiated timeout = 10000
[junit4:junit4]   2> 136 T1745 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@173654d0 name:ZooKeeperConnection Watcher:127.0.0.1:14566/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 137 T1736 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 137 T1736 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 147 T1736 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 149 T1736 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 152 T1736 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 160 T1736 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 161 T1736 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 175 T1736 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 176 T1736 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 280 T1736 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 281 T1736 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 284 T1736 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 285 T1736 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 295 T1736 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 296 T1736 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 298 T1736 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 299 T1736 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 302 T1736 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 303 T1736 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 306 T1736 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 306 T1736 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 309 T1736 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 310 T1736 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 317 T1741 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c95b6bc020001
[junit4:junit4]   2> 318 T1745 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 318 T1736 oaz.ZooKeeper.close Session: 0x13c95b6bc020001 closed
[junit4:junit4]   2> 318 T1738 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:24728 which had sessionid 0x13c95b6bc020001
[junit4:junit4]   2> 485 T1736 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 496 T1736 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:39493
[junit4:junit4]   2> 497 T1736 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 497 T1736 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 498 T1736 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359721446707
[junit4:junit4]   2> 498 T1736 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359721446707/solr.xml
[junit4:junit4]   2> 499 T1736 oasc.CoreContainer.<init> New CoreContainer 339734114
[junit4:junit4]   2> 500 T1736 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359721446707/'
[junit4:junit4]   2> 500 T1736 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359721446707/'
[junit4:junit4]   2> 556 T1736 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 556 T1736 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 557 T1736 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 557 T1736 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 558 T1736 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 558 T1736 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 559 T1736 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 559 T1736 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 559 T1736 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 560 T1736 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 581 T1736 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 601 T1736 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:14566/solr
[junit4:junit4]   2> 601 T1736 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 602 T1736 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:14566 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@3c659eaa
[junit4:junit4]   2> 603 T1736 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 603 T1755 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:14566. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 604 T1755 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:14566, initiating session
[junit4:junit4]   2> 605 T1738 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:23643
[junit4:junit4]   2> 605 T1738 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:23643
[junit4:junit4]   2> 606 T1740 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c95b6bc020002 with negotiated timeout 20000 for client /140.211.11.196:23643
[junit4:junit4]   2> 606 T1755 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:14566, sessionid = 0x13c95b6bc020002, negotiated timeout = 20000
[junit4:junit4]   2> 607 T1756 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3c659eaa name:ZooKeeperConnection Watcher:127.0.0.1:14566 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 607 T1736 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 608 T1741 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c95b6bc020002
[junit4:junit4]   2> 609 T1756 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 609 T1738 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:23643 which had sessionid 0x13c95b6bc020002
[junit4:junit4]   2> 609 T1736 oaz.ZooKeeper.close Session: 0x13c95b6bc020002 closed
[junit4:junit4]   2> 610 T1736 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 622 T1736 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:14566/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@7650b524
[junit4:junit4]   2> 623 T1736 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 623 T1757 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:14566. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 623 T1757 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:14566, initiating session
[junit4:junit4]   2> 624 T1738 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:20752
[junit4:junit4]   2> 624 T1738 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:20752
[junit4:junit4]   2> 625 T1740 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c95b6bc020003 with negotiated timeout 20000 for client /140.211.11.196:20752
[junit4:junit4]   2> 625 T1757 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:14566, sessionid = 0x13c95b6bc020003, negotiated timeout = 20000
[junit4:junit4]   2> 626 T1758 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7650b524 name:ZooKeeperConnection Watcher:127.0.0.1:14566/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 626 T1736 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 628 T1741 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c95b6bc020003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 636 T1736 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 638 T1736 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39493_o_z
[junit4:junit4]   2> 639 T1741 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c95b6bc020003 type:delete cxid:0x7 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:39493_o_z Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:39493_o_z
[junit4:junit4]   2> 640 T1736 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:39493_o_z
[junit4:junit4]   2> 648 T1736 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 657 T1741 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c95b6bc020003 type:delete cxid:0x15 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 664 T1736 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 666 T1736 oasc.Overseer.start Overseer (id=89110704711401475-127.0.0.1:39493_o_z-n_0000000000) starting
[junit4:junit4]   2> 667 T1741 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c95b6bc020003 type:create cxid:0x1a zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 668 T1741 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c95b6bc020003 type:create cxid:0x1b zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 670 T1741 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c95b6bc020003 type:create cxid:0x1c zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 679 T1741 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c95b6bc020003 type:create cxid:0x1d zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 680 T1760 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 681 T1736 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 689 T1736 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 690 T1759 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 695 T1761 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359721446707/collection1
[junit4:junit4]   2> 696 T1761 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 696 T1761 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 696 T1761 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 698 T1761 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359721446707/collection1/'
[junit4:junit4]   2> 699 T1761 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359721446707/collection1/lib/README' to classloader
[junit4:junit4]   2> 699 T1761 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359721446707/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 770 T1761 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 839 T1761 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 940 T1761 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 946 T1761 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 1541 T1761 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1551 T1761 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1555 T1761 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1583 T1761 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1588 T1761 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1592 T1761 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1593 T1761 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1594 T1761 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359721446707/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/control/data/
[junit4:junit4]   2> 1595 T1761 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@d278829
[junit4:junit4]   2> 1595 T1761 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 1596 T1761 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/control/data forceNew:false
[junit4:junit4]   2> 1597 T1761 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/control/data
[junit4:junit4]   2> 1597 T1761 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/control/data/index/
[junit4:junit4]   2> 1598 T1761 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1599 T1761 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/control/data/index forceNew:false
[junit4:junit4]   2> 1606 T1761 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3db8228a; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 1607 T1761 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 1608 T1761 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/control/data/index
[junit4:junit4]   2> 1608 T1761 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 1610 T1761 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1610 T1761 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 1611 T1761 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1611 T1761 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 1612 T1761 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1612 T1761 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1612 T1761 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1613 T1761 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1613 T1761 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1614 T1761 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 1614 T1761 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 1615 T1761 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1615 T1761 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 1615 T1761 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 1616 T1761 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 1616 T1761 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1617 T1761 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1617 T1761 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1618 T1761 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1618 T1761 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1619 T1761 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1619 T1761 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1620 T1761 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1620 T1761 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1621 T1761 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 1621 T1761 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 1634 T1761 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1639 T1761 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 1640 T1761 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 1642 T1761 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/control/data
[junit4:junit4]   2> 1644 T1761 oass.SolrIndexSearcher.<init> Opening Searcher@1d3d0872 main
[junit4:junit4]   2> 1645 T1761 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1645 T1761 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1646 T1761 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 1662 T1761 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 1713 T1762 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1d3d0872 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1717 T1761 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 1718 T1761 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 1718 T1741 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c95b6bc020003 type:create cxid:0x50 zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 2193 T1759 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2193 T1759 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:39493_o_z",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:39493/o_z"}
[junit4:junit4]   2> 2194 T1759 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4]   2> 2194 T1759 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 2194 T1741 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c95b6bc020003 type:create cxid:0x58 zxid:0x2c txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 2210 T1758 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 2729 T1761 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 2729 T1761 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:39493/o_z collection:control_collection shard:shard1
[junit4:junit4]   2> 2730 T1761 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4]   2> 2748 T1741 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c95b6bc020003 type:delete cxid:0x72 zxid:0x36 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4]   2> 2748 T1761 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 2749 T1741 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c95b6bc020003 type:create cxid:0x73 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2751 T1761 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 2751 T1761 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 2751 T1761 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:39493/o_z/collection1/
[junit4:junit4]   2> 2751 T1761 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 2752 T1761 oasc.SyncStrategy.syncToMe http://127.0.0.1:39493/o_z/collection1/ has no replicas
[junit4:junit4]   2> 2752 T1761 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:39493/o_z/collection1/
[junit4:junit4]   2> 2752 T1761 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4]   2> 2755 T1741 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c95b6bc020003 type:create cxid:0x7d zxid:0x3b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3714 T1759 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3737 T1758 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 3777 T1761 oasc.ZkController.register We are http://127.0.0.1:39493/o_z/collection1/ and leader is http://127.0.0.1:39493/o_z/collection1/
[junit4:junit4]   2> 3777 T1761 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:39493/o_z
[junit4:junit4]   2> 3777 T1761 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 3777 T1761 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 3778 T1761 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 3780 T1761 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3781 T1736 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 3781 T1736 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 3782 T1736 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 3796 T1736 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 3805 T1736 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:14566/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@448cc17b
[junit4:junit4]   2> 3806 T1736 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 3806 T1763 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:14566. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 3807 T1763 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:14566, initiating session
[junit4:junit4]   2> 3807 T1738 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:20743
[junit4:junit4]   2> 3807 T1738 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:20743
[junit4:junit4]   2> 3809 T1740 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c95b6bc020004 with negotiated timeout 10000 for client /140.211.11.196:20743
[junit4:junit4]   2> 3809 T1763 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:14566, sessionid = 0x13c95b6bc020004, negotiated timeout = 10000
[junit4:junit4]   2> 3810 T1764 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@448cc17b name:ZooKeeperConnection Watcher:127.0.0.1:14566/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 3810 T1736 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 3811 T1736 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 3812 T1736 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 3980 T1736 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 3983 T1736 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:43501
[junit4:junit4]   2> 3984 T1736 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 3985 T1736 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 3985 T1736 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359721450201
[junit4:junit4]   2> 3986 T1736 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359721450201/solr.xml
[junit4:junit4]   2> 3986 T1736 oasc.CoreContainer.<init> New CoreContainer 766582397
[junit4:junit4]   2> 3987 T1736 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359721450201/'
[junit4:junit4]   2> 3987 T1736 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359721450201/'
[junit4:junit4]   2> 4043 T1736 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 4044 T1736 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 4044 T1736 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 4045 T1736 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 4045 T1736 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 4046 T1736 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 4046 T1736 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 4046 T1736 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 4047 T1736 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 4047 T1736 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 4067 T1736 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 4086 T1736 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:14566/solr
[junit4:junit4]   2> 4087 T1736 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 4087 T1736 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:14566 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@4e6cf3b
[junit4:junit4]   2> 4088 T1736 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4089 T1774 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:14566. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4090 T1774 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:14566, initiating session
[junit4:junit4]   2> 4090 T1738 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:50425
[junit4:junit4]   2> 4090 T1738 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:50425
[junit4:junit4]   2> 4092 T1740 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c95b6bc020005 with negotiated timeout 20000 for client /140.211.11.196:50425
[junit4:junit4]   2> 4092 T1774 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:14566, sessionid = 0x13c95b6bc020005, negotiated timeout = 20000
[junit4:junit4]   2> 4093 T1775 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4e6cf3b name:ZooKeeperConnection Watcher:127.0.0.1:14566 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4093 T1736 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4094 T1741 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c95b6bc020005
[junit4:junit4]   2> 4104 T1775 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 4104 T1738 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:50425 which had sessionid 0x13c95b6bc020005
[junit4:junit4]   2> 4103 T1736 oaz.ZooKeeper.close Session: 0x13c95b6bc020005 closed
[junit4:junit4]   2> 4104 T1736 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 4114 T1736 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:14566/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@10094be0
[junit4:junit4]   2> 4115 T1736 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4115 T1776 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:14566. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4116 T1776 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:14566, initiating session
[junit4:junit4]   2> 4116 T1738 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:19101
[junit4:junit4]   2> 4116 T1738 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:19101
[junit4:junit4]   2> 4118 T1740 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c95b6bc020006 with negotiated timeout 20000 for client /140.211.11.196:19101
[junit4:junit4]   2> 4118 T1776 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:14566, sessionid = 0x13c95b6bc020006, negotiated timeout = 20000
[junit4:junit4]   2> 4118 T1777 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@10094be0 name:ZooKeeperConnection Watcher:127.0.0.1:14566/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4118 T1736 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4119 T1741 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c95b6bc020006 type:create cxid:0x1 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4120 T1741 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c95b6bc020006 type:create cxid:0x2 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4133 T1736 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43501_o_z
[junit4:junit4]   2> 4134 T1741 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c95b6bc020006 type:delete cxid:0x4 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:43501_o_z Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:43501_o_z
[junit4:junit4]   2> 4138 T1736 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:43501_o_z
[junit4:junit4]   2> 4184 T1758 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 4185 T1764 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 4185 T1758 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 4216 T1736 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 4223 T1778 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359721450201/collection1
[junit4:junit4]   2> 4223 T1778 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 4224 T1778 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 4224 T1778 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 4225 T1778 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359721450201/collection1/'
[junit4:junit4]   2> 4226 T1778 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359721450201/collection1/lib/README' to classloader
[junit4:junit4]   2> 4227 T1778 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359721450201/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 4280 T1778 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 4350 T1778 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 4451 T1778 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 4457 T1778 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 5068 T1778 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 5079 T1778 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 5083 T1778 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 5096 T1778 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5101 T1778 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5105 T1778 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5106 T1778 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5107 T1778 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359721450201/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty1/
[junit4:junit4]   2> 5108 T1778 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@d278829
[junit4:junit4]   2> 5108 T1778 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 5109 T1778 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty1 forceNew:false
[junit4:junit4]   2> 5110 T1778 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty1
[junit4:junit4]   2> 5110 T1778 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty1/index/
[junit4:junit4]   2> 5111 T1778 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 5112 T1778 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty1/index forceNew:false
[junit4:junit4]   2> 5118 T1778 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7f3f4f12; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 5119 T1778 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 5120 T1778 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty1/index
[junit4:junit4]   2> 5120 T1778 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 5122 T1778 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 5123 T1778 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 5123 T1778 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 5123 T1778 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 5124 T1778 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 5124 T1778 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 5125 T1778 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 5125 T1778 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 5126 T1778 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 5127 T1778 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 5127 T1778 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 5128 T1778 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 5128 T1778 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 5128 T1778 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 5129 T1778 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 5129 T1778 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 5130 T1778 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5131 T1778 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5131 T1778 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5132 T1778 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5132 T1778 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5133 T1778 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5134 T1778 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5134 T1778 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5135 T1778 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 5135 T1778 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 5148 T1778 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 5153 T1778 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 5154 T1778 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 5156 T1778 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty1
[junit4:junit4]   2> 5158 T1778 oass.SolrIndexSearcher.<init> Opening Searcher@40dbe769 main
[junit4:junit4]   2> 5159 T1778 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 5159 T1778 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 5160 T1778 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 5167 T1778 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 5216 T1779 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@40dbe769 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 5222 T1778 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 5222 T1778 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 5251 T1759 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 5251 T1759 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:39493_o_z",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:39493/o_z"}
[junit4:junit4]   2> 5255 T1759 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:43501_o_z",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:43501/o_z"}
[junit4:junit4]   2> 5256 T1759 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4]   2> 5256 T1759 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 5268 T1758 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5268 T1777 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5268 T1764 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6225 T1778 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 6225 T1778 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:43501/o_z collection:collection1 shard:shard1
[junit4:junit4]   2> 6226 T1778 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 6241 T1741 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c95b6bc020006 type:delete cxid:0x43 zxid:0x5a txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 6242 T1778 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 6242 T1741 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c95b6bc020006 type:create cxid:0x44 zxid:0x5b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6244 T1778 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 6244 T1778 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 6245 T1778 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:43501/o_z/collection1/
[junit4:junit4]   2> 6245 T1778 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 6245 T1778 oasc.SyncStrategy.syncToMe http://127.0.0.1:43501/o_z/collection1/ has no replicas
[junit4:junit4]   2> 6245 T1778 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:43501/o_z/collection1/
[junit4:junit4]   2> 6246 T1778 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 6249 T1741 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c95b6bc020006 type:create cxid:0x4e zxid:0x5f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6778 T1759 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6795 T1764 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6795 T1777 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6795 T1758 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6811 T1778 oasc.ZkController.register We are http://127.0.0.1:43501/o_z/collection1/ and leader is http://127.0.0.1:43501/o_z/collection1/
[junit4:junit4]   2> 6811 T1778 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:43501/o_z
[junit4:junit4]   2> 6811 T1778 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 6811 T1778 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 6812 T1778 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 6814 T1778 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6815 T1736 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 6815 T1736 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 6816 T1736 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 6999 T1736 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 7002 T1736 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:42630
[junit4:junit4]   2> 7003 T1736 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 7004 T1736 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 7004 T1736 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359721453218
[junit4:junit4]   2> 7005 T1736 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359721453218/solr.xml
[junit4:junit4]   2> 7005 T1736 oasc.CoreContainer.<init> New CoreContainer 336759368
[junit4:junit4]   2> 7006 T1736 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359721453218/'
[junit4:junit4]   2> 7006 T1736 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359721453218/'
[junit4:junit4]   2> 7062 T1736 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 7063 T1736 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 7063 T1736 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 7064 T1736 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 7064 T1736 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 7065 T1736 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 7065 T1736 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 7066 T1736 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 7066 T1736 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 7067 T1736 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 7087 T1736 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 7106 T1736 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:14566/solr
[junit4:junit4]   2> 7107 T1736 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 7107 T1736 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:14566 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@295f057c
[junit4:junit4]   2> 7108 T1736 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 7109 T1789 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:14566. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 7110 T1789 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:14566, initiating session
[junit4:junit4]   2> 7110 T1738 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:31826
[junit4:junit4]   2> 7110 T1738 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:31826
[junit4:junit4]   2> 7112 T1740 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c95b6bc020007 with negotiated timeout 20000 for client /140.211.11.196:31826
[junit4:junit4]   2> 7112 T1789 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:14566, sessionid = 0x13c95b6bc020007, negotiated timeout = 20000
[junit4:junit4]   2> 7112 T1790 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@295f057c name:ZooKeeperConnection Watcher:127.0.0.1:14566 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 7113 T1736 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 7114 T1741 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c95b6bc020007
[junit4:junit4]   2> 7124 T1790 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 7124 T1738 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:31826 which had sessionid 0x13c95b6bc020007
[junit4:junit4]   2> 7124 T1736 oaz.ZooKeeper.close Session: 0x13c95b6bc020007 closed
[junit4:junit4]   2> 7124 T1736 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 7134 T1736 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:14566/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@40328f60
[junit4:junit4]   2> 7135 T1736 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 7136 T1791 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:14566. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 7136 T1791 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:14566, initiating session
[junit4:junit4]   2> 7136 T1738 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:35702
[junit4:junit4]   2> 7137 T1738 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:35702
[junit4:junit4]   2> 7138 T1740 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c95b6bc020008 with negotiated timeout 20000 for client /140.211.11.196:35702
[junit4:junit4]   2> 7138 T1791 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:14566, sessionid = 0x13c95b6bc020008, negotiated timeout = 20000
[junit4:junit4]   2> 7138 T1792 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@40328f60 name:ZooKeeperConnection Watcher:127.0.0.1:14566/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 7139 T1736 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 7139 T1741 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c95b6bc020008 type:create cxid:0x1 zxid:0x6c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7141 T1741 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c95b6bc020008 type:create cxid:0x2 zxid:0x6d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7148 T1736 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42630_o_z
[junit4:junit4]   2> 7149 T1741 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c95b6bc020008 type:delete cxid:0x4 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:42630_o_z Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:42630_o_z
[junit4:junit4]   2> 7150 T1736 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:42630_o_z
[junit4:junit4]   2> 7152 T1777 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7152 T1764 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7152 T1758 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7153 T1758 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 7160 T1777 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7160 T1764 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7161 T1736 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 7168 T1793 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359721453218/collection1
[junit4:junit4]   2> 7168 T1793 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 7169 T1793 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 7169 T1793 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 7170 T1793 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359721453218/collection1/'
[junit4:junit4]   2> 7171 T1793 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359721453218/collection1/lib/README' to classloader
[junit4:junit4]   2> 7172 T1793 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359721453218/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 7225 T1793 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 7296 T1793 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 7397 T1793 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 7403 T1793 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 8039 T1793 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 8051 T1793 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 8055 T1793 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 8068 T1793 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8073 T1793 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 8077 T1793 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8078 T1793 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 8080 T1793 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359721453218/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty2/
[junit4:junit4]   2> 8080 T1793 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@d278829
[junit4:junit4]   2> 8080 T1793 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 8082 T1793 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty2 forceNew:false
[junit4:junit4]   2> 8083 T1793 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty2
[junit4:junit4]   2> 8083 T1793 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty2/index/
[junit4:junit4]   2> 8084 T1793 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 8085 T1793 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty2/index forceNew:false
[junit4:junit4]   2> 8092 T1793 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1e38d5eb; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 8092 T1793 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 8093 T1793 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty2/index
[junit4:junit4]   2> 8094 T1793 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 8096 T1793 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 8096 T1793 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 8097 T1793 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 8097 T1793 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 8098 T1793 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 8098 T1793 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 8098 T1793 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 8099 T1793 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 8100 T1793 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 8100 T1793 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 8101 T1793 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 8101 T1793 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 8102 T1793 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 8102 T1793 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 8103 T1793 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 8103 T1793 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 8104 T1793 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8104 T1793 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8105 T1793 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8105 T1793 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8106 T1793 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8107 T1793 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8107 T1793 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8108 T1793 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 8108 T1793 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 8109 T1793 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 8123 T1793 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 8128 T1793 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 8129 T1793 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 8132 T1793 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty2
[junit4:junit4]   2> 8134 T1793 oass.SolrIndexSearcher.<init> Opening Searcher@7f2b58a2 main
[junit4:junit4]   2> 8135 T1793 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 8135 T1793 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 8136 T1793 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 8143 T1793 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 8195 T1794 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7f2b58a2 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 8200 T1793 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 8201 T1793 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 8300 T1759 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8301 T1759 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:43501_o_z",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:43501/o_z"}
[junit4:junit4]   2> 8304 T1759 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:42630_o_z",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:42630/o_z"}
[junit4:junit4]   2> 8305 T1759 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 8305 T1759 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 8322 T1758 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8322 T1792 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8322 T1777 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8322 T1764 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9203 T1793 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 9203 T1793 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:42630/o_z collection:collection1 shard:shard2
[junit4:junit4]   2> 9204 T1793 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4]   2> 9224 T1741 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c95b6bc020008 type:delete cxid:0x42 zxid:0x7c txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
[junit4:junit4]   2> 9224 T1793 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 9225 T1741 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c95b6bc020008 type:create cxid:0x43 zxid:0x7d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9279 T1793 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 9279 T1793 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 9280 T1793 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:42630/o_z/collection1/
[junit4:junit4]   2> 9280 T1793 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 9280 T1793 oasc.SyncStrategy.syncToMe http://127.0.0.1:42630/o_z/collection1/ has no replicas
[junit4:junit4]   2> 9280 T1793 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:42630/o_z/collection1/
[junit4:junit4]   2> 9281 T1793 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4]   2> 9321 T1741 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c95b6bc020008 type:create cxid:0x4c zxid:0x80 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9827 T1759 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9887 T1758 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9887 T1792 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9887 T1777 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9887 T1764 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9900 T1793 oasc.ZkController.register We are http://127.0.0.1:42630/o_z/collection1/ and leader is http://127.0.0.1:42630/o_z/collection1/
[junit4:junit4]   2> 9900 T1793 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:42630/o_z
[junit4:junit4]   2> 9900 T1793 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 9900 T1793 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 9901 T1793 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 9902 T1793 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9903 T1736 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 9904 T1736 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 9904 T1736 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 10088 T1736 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 10092 T1736 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:49254
[junit4:junit4]   2> 10092 T1736 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 10093 T1736 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 10093 T1736 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359721456308
[junit4:junit4]   2> 10094 T1736 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359721456308/solr.xml
[junit4:junit4]   2> 10094 T1736 oasc.CoreContainer.<init> New CoreContainer 1148546086
[junit4:junit4]   2> 10095 T1736 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359721456308/'
[junit4:junit4]   2> 10096 T1736 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359721456308/'
[junit4:junit4]   2> 10151 T1736 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 10152 T1736 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 10152 T1736 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 10153 T1736 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 10153 T1736 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 10154 T1736 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 10154 T1736 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 10155 T1736 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 10155 T1736 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 10156 T1736 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 10177 T1736 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 10196 T1736 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:14566/solr
[junit4:junit4]   2> 10197 T1736 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 10198 T1736 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:14566 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@5230be7f
[junit4:junit4]   2> 10199 T1736 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 10199 T1804 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:14566. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 10200 T1804 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:14566, initiating session
[junit4:junit4]   2> 10200 T1738 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:39579
[junit4:junit4]   2> 10201 T1738 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:39579
[junit4:junit4]   2> 10202 T1740 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c95b6bc020009 with negotiated timeout 20000 for client /140.211.11.196:39579
[junit4:junit4]   2> 10203 T1804 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:14566, sessionid = 0x13c95b6bc020009, negotiated timeout = 20000
[junit4:junit4]   2> 10203 T1805 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5230be7f name:ZooKeeperConnection Watcher:127.0.0.1:14566 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 10203 T1736 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 10204 T1741 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c95b6bc020009
[junit4:junit4]   2> 10218 T1805 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 10218 T1738 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:39579 which had sessionid 0x13c95b6bc020009
[junit4:junit4]   2> 10218 T1736 oaz.ZooKeeper.close Session: 0x13c95b6bc020009 closed
[junit4:junit4]   2> 10219 T1736 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 10230 T1736 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:14566/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@b7baeb1
[junit4:junit4]   2> 10231 T1736 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 10232 T1806 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:14566. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 10232 T1806 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:14566, initiating session
[junit4:junit4]   2> 10232 T1738 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:32929
[junit4:junit4]   2> 10233 T1738 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:32929
[junit4:junit4]   2> 10234 T1740 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c95b6bc02000a with negotiated timeout 20000 for client /140.211.11.196:32929
[junit4:junit4]   2> 10234 T1806 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:14566, sessionid = 0x13c95b6bc02000a, negotiated timeout = 20000
[junit4:junit4]   2> 10234 T1807 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b7baeb1 name:ZooKeeperConnection Watcher:127.0.0.1:14566/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 10234 T1736 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 10235 T1741 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c95b6bc02000a type:create cxid:0x1 zxid:0x8d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 10237 T1741 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c95b6bc02000a type:create cxid:0x2 zxid:0x8e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 10244 T1736 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49254_o_z
[junit4:junit4]   2> 10245 T1741 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c95b6bc02000a type:delete cxid:0x4 zxid:0x8f txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:49254_o_z Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:49254_o_z
[junit4:junit4]   2> 10246 T1736 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49254_o_z
[junit4:junit4]   2> 10248 T1792 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10248 T1777 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10248 T1764 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10249 T1758 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 10249 T1758 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 10257 T1792 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 10257 T1777 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 10257 T1764 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 10258 T1736 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 10265 T1808 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359721456308/collection1
[junit4:junit4]   2> 10265 T1808 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 10266 T1808 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 10266 T1808 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 10267 T1808 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359721456308/collection1/'
[junit4:junit4]   2> 10268 T1808 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359721456308/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 10269 T1808 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359721456308/collection1/lib/README' to classloader
[junit4:junit4]   2> 10321 T1808 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 10391 T1808 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 10492 T1808 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 10498 T1808 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 11110 T1808 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 11121 T1808 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 11125 T1808 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 11137 T1808 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 11142 T1808 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 11146 T1808 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 11147 T1808 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 11148 T1808 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359721456308/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty3/
[junit4:junit4]   2> 11149 T1808 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@d278829
[junit4:junit4]   2> 11149 T1808 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 11150 T1808 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty3 forceNew:false
[junit4:junit4]   2> 11151 T1808 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty3
[junit4:junit4]   2> 11151 T1808 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty3/index/
[junit4:junit4]   2> 11152 T1808 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 11153 T1808 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty3/index forceNew:false
[junit4:junit4]   2> 11205 T1808 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@43bc926b; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 11205 T1808 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 11206 T1808 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty3/index
[junit4:junit4]   2> 11207 T1808 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 11209 T1808 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 11209 T1808 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 11209 T1808 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 11210 T1808 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 11210 T1808 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 11211 T1808 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 11211 T1808 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 11212 T1808 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 11212 T1808 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 11213 T1808 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 11214 T1808 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 11214 T1808 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 11215 T1808 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 11215 T1808 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 11215 T1808 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 11216 T1808 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 11216 T1808 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11217 T1808 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11217 T1808 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11218 T1808 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11219 T1808 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11219 T1808 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11220 T1808 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11220 T1808 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11221 T1808 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 11221 T1808 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 11234 T1808 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 11239 T1808 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 11240 T1808 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 11243 T1808 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty3
[junit4:junit4]   2> 11244 T1808 oass.SolrIndexSearcher.<init> Opening Searcher@212ad704 main
[junit4:junit4]   2> 11245 T1808 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 11246 T1808 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 11246 T1808 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 11253 T1808 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 11309 T1809 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@212ad704 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 11315 T1808 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 11315 T1808 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 11402 T1759 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11403 T1759 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:42630_o_z",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:42630/o_z"}
[junit4:junit4]   2> 11406 T1759 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:49254_o_z",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:49254/o_z"}
[junit4:junit4]   2> 11406 T1759 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 11406 T1759 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 11418 T1758 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11418 T1777 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11418 T1764 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11418 T1792 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11418 T1807 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12318 T1808 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 12318 T1808 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:49254/o_z collection:collection1 shard:shard1
[junit4:junit4]   2> 12321 T1808 oasc.ZkController.register We are http://127.0.0.1:49254/o_z/collection1/ and leader is http://127.0.0.1:43501/o_z/collection1/
[junit4:junit4]   2> 12321 T1808 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49254/o_z
[junit4:junit4]   2> 12321 T1808 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 12322 T1808 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C84 name=collection1 org.apache.solr.core.SolrCore@229355bd url=http://127.0.0.1:49254/o_z/collection1 node=127.0.0.1:49254_o_z C84_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:49254_o_z, base_url=http://127.0.0.1:49254/o_z}
[junit4:junit4]   2> 12322 T1810 C84 P49254 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 12322 T1808 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12323 T1810 C84 P49254 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 12324 T1810 C84 P49254 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 12324 T1810 C84 P49254 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 12324 T1736 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 12325 T1736 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 12325 T1810 C84 P49254 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 12326 T1736 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 12518 T1736 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 12521 T1736 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:34583
[junit4:junit4]   2> 12521 T1736 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 12522 T1736 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 12522 T1736 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359721458736
[junit4:junit4]   2> 12523 T1736 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359721458736/solr.xml
[junit4:junit4]   2> 12523 T1736 oasc.CoreContainer.<init> New CoreContainer 191945579
[junit4:junit4]   2> 12524 T1736 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359721458736/'
[junit4:junit4]   2> 12525 T1736 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359721458736/'
[junit4:junit4]   2> 12581 T1736 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 12582 T1736 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 12582 T1736 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 12583 T1736 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 12583 T1736 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 12583 T1736 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 12584 T1736 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 12584 T1736 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 12585 T1736 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 12585 T1736 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 12607 T1736 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 12627 T1736 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:14566/solr
[junit4:junit4]   2> 12627 T1736 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 12628 T1736 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:14566 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@679f491b
[junit4:junit4]   2> 12629 T1736 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 12629 T1820 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:14566. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 12631 T1820 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:14566, initiating session
[junit4:junit4]   2> 12631 T1738 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:51035
[junit4:junit4]   2> 12632 T1738 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:51035
[junit4:junit4]   2> 12634 T1740 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c95b6bc02000b with negotiated timeout 20000 for client /140.211.11.196:51035
[junit4:junit4]   2> 12634 T1820 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:14566, sessionid = 0x13c95b6bc02000b, negotiated timeout = 20000
[junit4:junit4]   2> 12634 T1821 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@679f491b name:ZooKeeperConnection Watcher:127.0.0.1:14566 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 12634 T1736 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 12635 T1741 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c95b6bc02000b
[junit4:junit4]   2> 12647 T1821 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 12647 T1738 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:51035 which had sessionid 0x13c95b6bc02000b
[junit4:junit4]   2> 12647 T1736 oaz.ZooKeeper.close Session: 0x13c95b6bc02000b closed
[junit4:junit4]   2> 12648 T1736 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 12660 T1736 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:14566/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@51ec4ecf
[junit4:junit4]   2> 12661 T1736 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 12661 T1822 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:14566. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 12661 T1822 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:14566, initiating session
[junit4:junit4]   2> 12662 T1738 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:27377
[junit4:junit4]   2> 12662 T1738 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:27377
[junit4:junit4]   2> 12663 T1740 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c95b6bc02000c with negotiated timeout 20000 for client /140.211.11.196:27377
[junit4:junit4]   2> 12663 T1822 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:14566, sessionid = 0x13c95b6bc02000c, negotiated timeout = 20000
[junit4:junit4]   2> 12664 T1823 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@51ec4ecf name:ZooKeeperConnection Watcher:127.0.0.1:14566/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 12664 T1736 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 12665 T1741 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c95b6bc02000c type:create cxid:0x1 zxid:0x9f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12666 T1741 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c95b6bc02000c type:create cxid:0x2 zxid:0xa0 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12674 T1736 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34583_o_z
[junit4:junit4]   2> 12674 T1741 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c95b6bc02000c type:delete cxid:0x4 zxid:0xa1 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:34583_o_z Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:34583_o_z
[junit4:junit4]   2> 12675 T1736 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:34583_o_z
[junit4:junit4]   2> 12677 T1807 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12677 T1764 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12677 T1777 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12677 T1792 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12678 T1758 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 12678 T1758 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 12679 T1807 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 12685 T1764 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 12685 T1792 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 12685 T1777 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 12686 T1736 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 12693 T1824 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359721458736/collection1
[junit4:junit4]   2> 12693 T1824 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 12694 T1824 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 12694 T1824 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 12695 T1824 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359721458736/collection1/'
[junit4:junit4]   2> 12696 T1824 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359721458736/collection1/lib/README' to classloader
[junit4:junit4]   2> 12697 T1824 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359721458736/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 12749 T1824 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 12819 T1824 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 12920 T1824 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 12926 T1824 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 12931 T1759 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12932 T1759 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:49254_o_z",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:49254/o_z"}
[junit4:junit4]   2> 12944 T1758 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 12944 T1764 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 12944 T1777 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 12944 T1823 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 12944 T1807 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 12944 T1792 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 13614 T1824 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 13625 T1824 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 13629 T1824 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 13641 T1824 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 13646 T1824 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 13650 T1824 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 13651 T1824 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 13652 T1824 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359721458736/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty4/
[junit4:junit4]   2> 13652 T1824 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@d278829
[junit4:junit4]   2> 13653 T1824 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 13654 T1824 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty4 forceNew:false
[junit4:junit4]   2> 13655 T1824 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty4
[junit4:junit4]   2> 13655 T1824 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty4/index/
[junit4:junit4]   2> 13655 T1824 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 13658 T1824 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty4/index forceNew:false
[junit4:junit4]   2> 13664 T1824 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@34b30; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 13664 T1824 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 13665 T1824 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty4/index
[junit4:junit4]   2> 13666 T1824 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 13668 T1824 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 13668 T1824 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 13668 T1824 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 13669 T1824 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 13670 T1824 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 13670 T1824 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 13670 T1824 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 13671 T1824 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 13671 T1824 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 13672 T1824 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 13673 T1824 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 13673 T1824 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 13674 T1824 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 13674 T1824 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 13674 T1824 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 13675 T1824 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 13675 T1824 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13676 T1824 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13677 T1824 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13677 T1824 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13678 T1824 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13678 T1824 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13679 T1824 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13679 T1824 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13680 T1824 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 13681 T1824 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 13693 T1824 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 13698 T1824 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 13699 T1824 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 13702 T1824 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty4
[junit4:junit4]   2> 13703 T1824 oass.SolrIndexSearcher.<init> Opening Searcher@1633ea9b main
[junit4:junit4]   2> 13704 T1824 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 13705 T1824 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 13705 T1824 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 13712 T1824 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 13764 T1825 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1633ea9b main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 13770 T1824 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 13770 T1824 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 14448 T1759 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14449 T1759 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:34583_o_z",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:34583/o_z"}
[junit4:junit4]   2> 14449 T1759 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 14449 T1759 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 14469 T1758 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14469 T1823 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14469 T1764 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14469 T1777 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14469 T1807 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14469 T1792 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14772 T1824 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 14772 T1824 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:34583/o_z collection:collection1 shard:shard2
[junit4:junit4]   2> 14774 T1824 oasc.ZkController.register We are http://127.0.0.1:34583/o_z/collection1/ and leader is http://127.0.0.1:42630/o_z/collection1/
[junit4:junit4]   2> 14775 T1824 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:34583/o_z
[junit4:junit4]   2> 14775 T1824 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 14775 T1824 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C85 name=collection1 org.apache.solr.core.SolrCore@1d793dc2 url=http://127.0.0.1:34583/o_z/collection1 node=127.0.0.1:34583_o_z C85_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:34583_o_z, base_url=http://127.0.0.1:34583/o_z}
[junit4:junit4]   2> 14776 T1826 C85 P34583 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 14776 T1824 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14777 T1826 C85 P34583 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 14777 T1826 C85 P34583 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 14777 T1826 C85 P34583 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 14777 T1736 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 14778 T1736 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 14778 T1826 C85 P34583 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 14779 T1736 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 14799 T1736 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14801 T1736 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4]   2> 14802 T1736 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C84_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:49254_o_z, base_url=http://127.0.0.1:49254/o_z}
[junit4:junit4]   2> 15351 T1810 C84 P49254 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:43501/o_z/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 15351 T1810 C84 P49254 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:49254/o_z START replicas=[http://127.0.0.1:43501/o_z/collection1/] nUpdates=100
[junit4:junit4]   2> 15352 T1810 C84 P49254 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 15353 T1810 C84 P49254 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 15353 T1810 C84 P49254 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 15353 T1810 C84 P49254 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 15353 T1810 C84 P49254 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 15353 T1810 C84 P49254 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:43501/o_z/collection1/. core=collection1
[junit4:junit4]   2> 15354 T1810 C84 P49254 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C86 name=collection1 org.apache.solr.core.SolrCore@7469bea5 url=http://127.0.0.1:43501/o_z/collection1 node=127.0.0.1:43501_o_z C86_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:43501_o_z, base_url=http://127.0.0.1:43501/o_z, leader=true}
[junit4:junit4]   2> 15364 T1768 C86 P43501 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15365 T1769 C86 P43501 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 15366 T1769 C86 P43501 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty1
[junit4:junit4]   2> 15371 T1769 C86 P43501 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7f3f4f12; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 15371 T1769 C86 P43501 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 15405 T1769 C86 P43501 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7f3f4f12; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7f3f4f12; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 15406 T1769 C86 P43501 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 15407 T1769 C86 P43501 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty1
[junit4:junit4]   2> 15408 T1769 C86 P43501 oass.SolrIndexSearcher.<init> Opening Searcher@76d56dba realtime
[junit4:junit4]   2> 15408 T1769 C86 P43501 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15409 T1769 C86 P43501 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 43
[junit4:junit4]   2> 15409 T1810 C84 P49254 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 15410 T1810 C84 P49254 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 15412 T1770 C86 P43501 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 15413 T1770 C86 P43501 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 15414 T1810 C84 P49254 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 15414 T1810 C84 P49254 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 15414 T1810 C84 P49254 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 15416 T1770 C86 P43501 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty1
[junit4:junit4]   2> 15417 T1770 C86 P43501 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty1/index
[junit4:junit4]   2> 15418 T1770 C86 P43501 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=2 
[junit4:junit4]   2> 15418 T1810 C84 P49254 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 15420 T1810 C84 P49254 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty3/index.20130201202421807 forceNew:false
[junit4:junit4]   2> 15420 T1810 C84 P49254 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty3
[junit4:junit4]   2> 15421 T1810 C84 P49254 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty3/index.20130201202421807 lockFactory=org.apache.lucene.store.NativeFSLockFactory@33dcb7d9; maxCacheMB=48.0 maxMergeSizeMB=4.0)) fullCopy=true
[junit4:junit4]   2> 15425 T1770 C86 P43501 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> 15427 T1810 C84 P49254 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty3/index
[junit4:junit4]   2> 15427 T1810 C84 P49254 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 15427 T1810 C84 P49254 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130201202421807
[junit4:junit4]   2> 15429 T1810 C84 P49254 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty3
[junit4:junit4]   2> 15429 T1810 C84 P49254 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty3
[junit4:junit4]   2> 15430 T1810 C84 P49254 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty3/index/ new=./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty3/index.20130201202421807
[junit4:junit4]   2> 15433 T1810 C84 P49254 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty3/index.20130201202421807 lockFactory=org.apache.lucene.store.NativeFSLockFactory@33dcb7d9; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 15434 T1810 C84 P49254 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 15435 T1810 C84 P49254 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty3
[junit4:junit4]   2> 15435 T1810 C84 P49254 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 15435 T1810 C84 P49254 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 15436 T1810 C84 P49254 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 15437 T1810 C84 P49254 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty3/index.20130201202421807
[junit4:junit4]   2> 15437 T1810 C84 P49254 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty3
[junit4:junit4]   2> 15441 T1810 C84 P49254 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty3/index.20130201202421807 lockFactory=org.apache.lucene.store.NativeFSLockFactory@33dcb7d9; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 15441 T1810 C84 P49254 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 15442 T1810 C84 P49254 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 15443 T1810 C84 P49254 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty3
[junit4:junit4]   2> 15444 T1810 C84 P49254 oass.SolrIndexSearcher.<init> Opening Searcher@dabe1b6 main
[junit4:junit4]   2> 15446 T1809 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@dabe1b6 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 15446 T1809 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty3/index
[junit4:junit4]   2> 15447 T1810 C84 P49254 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 15450 T1810 C84 P49254 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty3/index.20130201202421807 lockFactory=org.apache.lucene.store.NativeFSLockFactory@33dcb7d9; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty3/index.20130201202421807 lockFactory=org.apache.lucene.store.NativeFSLockFactory@33dcb7d9; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 15450 T1810 C84 P49254 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 15452 T1810 C84 P49254 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty3
[junit4:junit4]   2> 15452 T1810 C84 P49254 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15452 T1810 C84 P49254 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty3/index.20130201202421807
[junit4:junit4]   2> 15452 T1810 C84 P49254 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty3/index
[junit4:junit4]   2> 15452 T1810 C84 P49254 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 15453 T1810 C84 P49254 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 15453 T1810 C84 P49254 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 15453 T1810 C84 P49254 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 15459 T1810 C84 P49254 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 15803 T1736 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 15973 T1759 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 15974 T1759 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:34583_o_z",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:34583/o_z"}
[junit4:junit4]   2> 15978 T1759 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:49254_o_z",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:49254/o_z"}
[junit4:junit4]   2> 15996 T1792 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15996 T1758 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15996 T1807 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15996 T1777 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15996 T1764 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15996 T1823 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16805 T1736 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17807 T1736 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C85_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:34583_o_z, base_url=http://127.0.0.1:34583/o_z}
[junit4:junit4]   2> 18803 T1826 C85 P34583 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:42630/o_z/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 18803 T1826 C85 P34583 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:34583/o_z START replicas=[http://127.0.0.1:42630/o_z/collection1/] nUpdates=100
[junit4:junit4]   2> 18803 T1826 C85 P34583 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 18804 T1826 C85 P34583 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 18804 T1826 C85 P34583 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 18804 T1826 C85 P34583 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 18804 T1826 C85 P34583 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 18804 T1826 C85 P34583 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:42630/o_z/collection1/. core=collection1
[junit4:junit4]   2> 18805 T1826 C85 P34583 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 18809 T1736 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> ASYNC  NEW_CORE C87 name=collection1 org.apache.solr.core.SolrCore@70e83e5f url=http://127.0.0.1:42630/o_z/collection1 node=127.0.0.1:42630_o_z C87_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:42630_o_z, base_url=http://127.0.0.1:42630/o_z, leader=true}
[junit4:junit4]   2> 18817 T1783 C87 P42630 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1 
[junit4:junit4]   2> 18818 T1784 C87 P42630 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 18819 T1784 C87 P42630 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty2
[junit4:junit4]   2> 18822 T1784 C87 P42630 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1e38d5eb; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 18824 T1784 C87 P42630 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 18829 T1784 C87 P42630 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1e38d5eb; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1e38d5eb; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 18830 T1784 C87 P42630 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 18831 T1784 C87 P42630 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty2
[junit4:junit4]   2> 18832 T1784 C87 P42630 oass.SolrIndexSearcher.<init> Opening Searcher@63a0e656 realtime
[junit4:junit4]   2> 18832 T1784 C87 P42630 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 18832 T1784 C87 P42630 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 14
[junit4:junit4]   2> 18833 T1826 C85 P34583 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 18833 T1826 C85 P34583 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 18835 T1785 C87 P42630 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 18836 T1785 C87 P42630 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 18836 T1826 C85 P34583 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 18836 T1826 C85 P34583 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 18837 T1826 C85 P34583 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 18839 T1785 C87 P42630 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty2
[junit4:junit4]   2> 18840 T1785 C87 P42630 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty2/index
[junit4:junit4]   2> 18840 T1785 C87 P42630 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1 
[junit4:junit4]   2> 18841 T1826 C85 P34583 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 18842 T1826 C85 P34583 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty4/index.20130201202425229 forceNew:false
[junit4:junit4]   2> 18843 T1826 C85 P34583 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty4
[junit4:junit4]   2> 18843 T1826 C85 P34583 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty4/index.20130201202425229 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6135df82; maxCacheMB=48.0 maxMergeSizeMB=4.0)) fullCopy=true
[junit4:junit4]   2> 18847 T1785 C87 P42630 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> 18848 T1826 C85 P34583 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty4/index
[junit4:junit4]   2> 18848 T1826 C85 P34583 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 18849 T1826 C85 P34583 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130201202425229
[junit4:junit4]   2> 18850 T1826 C85 P34583 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty4
[junit4:junit4]   2> 18850 T1826 C85 P34583 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty4
[junit4:junit4]   2> 18850 T1826 C85 P34583 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty4/index/ new=./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty4/index.20130201202425229
[junit4:junit4]   2> 18853 T1826 C85 P34583 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty4/index.20130201202425229 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6135df82; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 18853 T1826 C85 P34583 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 18854 T1826 C85 P34583 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty4
[junit4:junit4]   2> 18854 T1826 C85 P34583 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 18854 T1826 C85 P34583 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 18854 T1826 C85 P34583 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 18855 T1826 C85 P34583 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty4/index.20130201202425229
[junit4:junit4]   2> 18855 T1826 C85 P34583 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty4
[junit4:junit4]   2> 18858 T1826 C85 P34583 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty4/index.20130201202425229 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6135df82; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 18858 T1826 C85 P34583 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 18858 T1826 C85 P34583 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 18858 T1826 C85 P34583 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty4
[junit4:junit4]   2> 18859 T1826 C85 P34583 oass.SolrIndexSearcher.<init> Opening Searcher@20d0ac06 main
[junit4:junit4]   2> 18861 T1825 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@20d0ac06 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 18862 T1825 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty4/index
[junit4:junit4]   2> 18862 T1826 C85 P34583 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 18865 T1826 C85 P34583 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty4/index.20130201202425229 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6135df82; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty4/index.20130201202425229 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6135df82; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 18865 T1826 C85 P34583 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 18866 T1826 C85 P34583 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty4
[junit4:junit4]   2> 18866 T1826 C85 P34583 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 18866 T1826 C85 P34583 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty4/index.20130201202425229
[junit4:junit4]   2> 18866 T1826 C85 P34583 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty4/index
[junit4:junit4]   2> 18867 T1826 C85 P34583 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 18867 T1826 C85 P34583 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 18867 T1826 C85 P34583 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 18867 T1826 C85 P34583 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 18875 T1826 C85 P34583 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 19004 T1759 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19005 T1759 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:34583_o_z",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:34583/o_z"}
[junit4:junit4]   2> 19018 T1758 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19018 T1807 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19018 T1777 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19018 T1764 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19018 T1792 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19018 T1823 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19811 T1736 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19812 T1736 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4]   2> ASYNC  NEW_CORE C88 name=collection1 org.apache.solr.core.SolrCore@32dd1d61 url=http://127.0.0.1:39493/o_z/collection1 node=127.0.0.1:39493_o_z C88_STATE=coll:control_collection core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:39493_o_z, base_url=http://127.0.0.1:39493/o_z, leader=true}
[junit4:junit4]   2> 19825 T1748 C88 P39493 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/control/data
[junit4:junit4]   2> 19830 T1748 C88 P39493 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3db8228a; maxCacheMB=48.0 maxMergeSizeMB=4.0)),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 19831 T1748 C88 P39493 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 19832 T1748 C88 P39493 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/control/data
[junit4:junit4]   2> 19833 T1748 C88 P39493 oass.SolrIndexSearcher.<init> Opening Searcher@3df7e232 realtime
[junit4:junit4]   2> 19834 T1748 C88 P39493 /update {wt=javabin&version=2} {deleteByQuery=*:* (-1425771296154714112)} 0 11
[junit4:junit4]   2> 19838 T1786 C87 P42630 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty2
[junit4:junit4]   2> 19843 T1771 C86 P43501 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty1
[junit4:junit4]   2>  C85_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:34583_o_z, base_url=http://127.0.0.1:34583/o_z}
[junit4:junit4]   2> 19851 T1813 C85 P34583 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty4
[junit4:junit4]   2> 19852 T1813 C85 P34583 /update {update.distrib=FROMLEADER&_version_=-1425771296167297024&update.from=http://127.0.0.1:42630/o_z/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1425771296167297024)} 0 3
[junit4:junit4]   2>  C84_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:49254_o_z, base_url=http://127.0.0.1:49254/o_z}
[junit4:junit4]   2> 19856 T1797 C84 P49254 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359721446394/jetty3
[junit4:junit4]   2> 19857 T1797 C84 P49254 /update {update.distrib=FROMLEADER&_version_=-1425771296172539904&update.from=http://127.0.0.1:43501/o_z/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1425771296172539904)} 0 4
[junit4:junit4]   2> 19857 T1771 C86 P43501 /update {update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1425771296172539904)} 0 16
[junit4:junit4]   2> 19858 T1786 C87 P42630 /update {wt=javabin&version=2} {deleteByQuery=*:* (-1425771296167297024)} 0 22
[junit4:junit4]   2> 19865 T1748 C88 P39493 /update {wt=javabin&version=2} {add=[1 (1425771296193511424)]} 0 4
[junit4:junit4]   2> 19881 T1797 C84 P49254 /update {distrib.from=http://127.0.0.1:43501/o_z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1425771296203997184)]} 0 4
[junit4:junit4]   2> 19882 T1771 C86 P43501 /update {distrib.from=http://127.0.0.1:42630/o_z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1425771296203997184)]} 0 11
[junit4:junit4]   2> 19882 T1786 C87 P42630 /update {wt=javabin&version=2} {add=[1]} 0 14
[junit4:junit4]   2> 19887 T1748 C88 P39493 /update {wt=javabin&version=2} {add=[2 (1425771296218677248)]} 0 2
[junit4:junit4]   2> 19900 T1813 C85 P34583 /update {distrib.from=http://127.0.0.1:42630/o_z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1425771296223920128)]} 0 4
[junit4:junit4]   2> 19900 T1787 C87 P42630 /update {distrib.from=http://127.0.0.1:43501/o_z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[2 (1425771296223920128)]} 0 10
[junit4:junit4]   2> 19901 T1772 C86 P43501 /update {wt=javabin&version=2} {add=[2]} 0 12
[junit4:junit4]   2> 19906 T1748 C88 P39493 /update {wt=javabin&version=2} {add=[3 (1425771296238600192)]} 0 2
[junit4:junit4]   2> 19915 T1813 C85 P34583 /update {distrib.from=http://127.0.0.1:42630/o_z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[3 (1425771296243843072)]} 0 2
[junit4:junit4]   2> 19916 T1786 C87 P42630 /update {wt=javabin&version=2} {add=[3 (1425771296243843072)]} 0 7
[junit4:junit4]   2> 19920 T1748 C88 P39493 /update {wt=javabin&version=2} {add=[4 (1425771296253280256)]} 0 2
[junit4:junit4]   2> 19930 T1797 C84 P49254 /update {distrib.from=http://127.0.0.1:43501/o_z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[4 (1425771296258523136)]} 0 2
[junit4:junit4]   2> 19931 T1772 C86 P43501 /update {wt=javabin&version=2} {add=[4 (1425771296258523136)]} 0 8
[junit4:junit4]   2> 19936 T1748 C88 P39493 /update {wt=javabin&version=2} {add=[5 (1425771296270057472)]} 0 2
[junit4:junit4]   2> 19944 T1813 C85 P34583 /update {distrib.from=http://127.0.0.1:42630/o_z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[5 (1425771296274251776)]} 0 2
[junit4:junit4]   2> 19945 T1786 C87 P42630 /update {wt=javabin&version=2} {add=[5 (1425771296274251776)]} 0 7
[junit4:junit4]   2> 19949 T1748 C88 P39493 /update {wt=javabin&version=2} {add=[6 (1425771296283688960)]} 0 2
[junit4:junit4]   2> 19957 T1813 C85 P34583 /update {distrib.from=http://127.0.0.1:42630/o_z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[6 (1425771296287883264)]} 0 2
[junit4:junit4]   2> 19957 T1786 C87 P42630 /update {wt=javabin&version=2} {add=[6 (1425771296287883264)]} 0 6
[junit4:junit4]   2> 19962 T1748 C88 P39493 /update {wt=javabin&version=2} {add=[7 (1425771296297320448)]} 0 2
[junit4:junit4]   2> 19972 T1813 C85 P34583 /update {distrib.from=http://127.0.0.1:42630/o_z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[7 (1425771296303611904)]} 0 2
[junit4:junit4]   2> 19972 T1787 C87 P42630 /update {distrib.from=http://127.0.0.1:43501/o_z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[7 (1425771296303611904)]} 0 6
[junit4:junit4]   2> 19973 T1772 C86 P43501 /update {wt=javabin&version=2} {add=[7]} 0 9
[junit4:junit4]   2> 19979 T1748 C88 P39493 /update {wt=javabin&version=2} {add=[8 (1425771296315146240)]} 0 3
[junit4:junit4]   2> 19990 T1797 C84 P49254 /update {distrib.from=http://127.0.0.1:43501/o_z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[8 (1425771296322486272)]} 0 2
[junit4:junit4]   2> 19991 T1772 C86 P43501 /update {wt=javabin&version=2} {add=[8 (1425771296322486272)]} 0 7
[junit4:junit4]   2> 19997 T1748 C88 P39493 /update {wt=javabin&version=2} {add=[9 (1425771296334020608)]} 0 2
[junit4:junit4]   2> 20006 T1813 C85 P34583 /update {distrib.from=http://127.0.0.1:42630/o_z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[9 (1425771296339263488)]} 0 2
[junit4:junit4]   2> 20007 T1786 C87 P42630 /update {wt=javabin&version=2} {add=[9 (1425771296339263488)]} 0 7
[junit4:junit4]   2> 20025 T1748 C88 P39493 /update {wt=javabin&version=2} {add=[10 (1425771296348700672)]} 0 16
[junit4:junit4]   2> 20034 T1797 C84 P49254 /update {distrib.from=http://127.0.0.1:43501/o_z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10 (1425771296368623616)]} 0 2
[junit4:junit4]   2> 20035 T1772 C86 P43501 /update {wt=javabin&version=2} {add=[10 (1425771296368623616)]} 0 8
[junit4:junit4]   2> 20041 T1748 C88 P39493 /update {wt=javabin&version=2} {add=[11 (1425771296379109376)]} 0 3
[junit4:junit4]   2> 20050 T1797 C84 P49254 /update {distrib.from=http://127.0.0.1:43501/o_z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[11 (1425771296385400832)]} 0 2
[junit4:junit4]   2> 20051 T1772 C86 P43501 /update {wt=javabin&version=2} {add=[11 (1425771296385400832)]} 0 7
[junit4:junit4]   2> 20056 T1748 C88 P39493 /update {wt=javabin&version=2} {add=[12 (1425771296395886592)]} 0 2
[junit4:junit4]   2> 20068 T1797 C84 P49254 /update {distrib.from=http://127.0.0.1:43501/o_z/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[12 (1425771296404275200)]} 0 2
[junit4:junit4]   2> 20069 T1771 C86 P43501 /update {distrib.from=http://127.0.0.1:42630/o_z/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[12 (1425771296404275200)]} 0 7
[junit4:junit4]   2> 20069 T1786 C87 P42630 /update {wt=javabin&version=2} {add=[12]} 0 10
[junit4:junit4]   2> 20075 T1748 C88 P39493 /update {wt=javabin&version=2} {add=[13 (1425771296415809536)]} 0 2
[junit4:junit4]   2> 20088 T1797 C84 P49254 /update {distr

[...truncated too long message...]

gQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  38) Thread[id=1746, name=HashSessionScavenger-50, state=TIMED_WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at java.lang.Object.wait(Native Method)
[junit4:junit4]   2> 	        at java.util.TimerThread.mainLoop(Timer.java:552)
[junit4:junit4]   2> 	        at java.util.TimerThread.run(Timer.java:505)
[junit4:junit4]   2> 	  39) Thread[id=1779, name=searcherExecutor-871-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  40) Thread[id=1885, name=searcherExecutor-951-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  41) Thread[id=1928, name=searcherExecutor-967-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 	  42) Thread[id=2157, name=searcherExecutor-1042-thread-1, state=WAITING, group=TGRP-BasicDistributedZkTest]
[junit4:junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
[junit4:junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
[junit4:junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
[junit4:junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> NOTE: test params are: codec=Lucene41: {n_f1=PostingsFormat(name=SimpleText), foo_b=PostingsFormat(name=NestedPulsing), cat=PostingsFormat(name=Lucene41WithOrds), foo_d=Pulsing41(freqCutoff=5 minBlockSize=3 maxBlockSize=76), foo_f=PostingsFormat(name=NestedPulsing), n_tl1=PostingsFormat(name=SimpleText), n_d1=PostingsFormat(name=Lucene41WithOrds), rnd_b=Pulsing41(freqCutoff=5 minBlockSize=3 maxBlockSize=76), intDefault=PostingsFormat(name=Lucene41WithOrds), n_td1=PostingsFormat(name=SimpleText), timestamp=PostingsFormat(name=Lucene41WithOrds), id=Pulsing41(freqCutoff=5 minBlockSize=3 maxBlockSize=76), range_facet_sl=Pulsing41(freqCutoff=5 minBlockSize=3 maxBlockSize=76), range_facet_si=PostingsFormat(name=Lucene41WithOrds), oddField_s=Pulsing41(freqCutoff=5 minBlockSize=3 maxBlockSize=76), sequence_i=Pulsing41(freqCutoff=5 minBlockSize=3 maxBlockSize=76), name=Pulsing41(freqCutoff=5 minBlockSize=3 maxBlockSize=76), foo_i=PostingsFormat(name=SimpleText), regex_dup_B_s=PostingsFormat(name=Lucene41WithOrds), multiDefault=PostingsFormat(name=Lucene41WithOrds), n_tf1=PostingsFormat(name=Lucene41WithOrds), n_dt1=PostingsFormat(name=SimpleText), genre_s=Pulsing41(freqCutoff=5 minBlockSize=3 maxBlockSize=76), author_t=PostingsFormat(name=Lucene41WithOrds), n_ti1=Pulsing41(freqCutoff=5 minBlockSize=3 maxBlockSize=76), range_facet_l=PostingsFormat(name=Lucene41WithOrds), text=PostingsFormat(name=NestedPulsing), _version_=PostingsFormat(name=SimpleText), val_i=Pulsing41(freqCutoff=5 minBlockSize=3 maxBlockSize=76), SubjectTerms_mfacet=PostingsFormat(name=SimpleText), series_t=PostingsFormat(name=Lucene41WithOrds), a_t=PostingsFormat(name=Lucene41WithOrds), n_tdt1=PostingsFormat(name=SimpleText), regex_dup_A_s=PostingsFormat(name=NestedPulsing), price=PostingsFormat(name=NestedPulsing), other_tl1=PostingsFormat(name=Lucene41WithOrds), n_l1=PostingsFormat(name=Lucene41WithOrds), a_si=PostingsFormat(name=SimpleText), inStock=PostingsFormat(name=NestedPulsing)}, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=ar_EG, timezone=Singapore
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=45,free=280441232,total=536870912
[junit4:junit4]   2> NOTE: All tests run in this JVM: [PathHierarchyTokenizerFactoryTest, DOMUtilTest, DirectUpdateHandlerTest, TestSolrXMLSerializer, CoreContainerCoreInitFailuresTest, UUIDFieldTest, TestReplicationHandler, TestCSVLoader, SoftAutoCommitTest, ClusterStateUpdateTest, ZkCLITest, TestDistributedSearch, SuggesterTSTTest, ClusterStateTest, OutputWriterTest, TimeZoneUtilsTest, SignatureUpdateProcessorFactoryTest, SolrCoreTest, TestSolrDeletionPolicy2, BasicFunctionalityTest, TestZkChroot, FieldMutatingUpdateProcessorTest, TestDocSet, PrimitiveFieldTypeTest, DistributedQueryElevationComponentTest, BasicDistributedZk2Test, OverseerTest, FullSolrCloudDistribCmdsTest, LukeRequestHandlerTest, TestRTGBase, TestSolr4Spatial, SuggesterTest, DocumentBuilderTest, SpellingQueryConverterTest, BadIndexSchemaTest, TestPropInject, TestArbitraryIndexDir, IndexReaderFactoryTest, TestSurroundQueryParser, XsltUpdateRequestHandlerTest, URLClassifyProcessorTest, TestLazyCores, ReturnFieldsTest, TestSolrDeletionPolicy1, SolrInfoMBeanTest, OpenExchangeRatesOrgProviderTest, ConvertedLegacyTest, TestComponentsName, DisMaxRequestHandlerTest, SchemaVersionSpecificBehaviorTest, PrimUtilsTest, StatelessScriptUpdateProcessorFactoryTest, TestQueryUtils, HighlighterTest, IndexSchemaTest, StatsComponentTest, QueryParsingTest, DirectSolrSpellCheckerTest, RequiredFieldsTest, BasicDistributedZkTest]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=BasicDistributedZkTest -Dtests.seed=FBCDCC7974E9B069 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ar_EG -Dtests.timezone=Singapore -Dtests.file.encoding=US-ASCII
[junit4:junit4] ERROR   0.00s J1 | BasicDistributedZkTest (suite) <<<
[junit4:junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([FBCDCC7974E9B069]:0)
[junit4:junit4] Completed on J1 in 7224.42s, 1 test, 2 errors <<< FAILURES!

[...truncated 11 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:346: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/common-build.xml:428: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:1176: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:841: There were test failures: 248 suites, 1037 tests, 1 suite-level error, 1 error, 23 ignored (5 assumptions)

Total time: 149 minutes 9 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-trunk-java7 - Build # 3697 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-java7/3697/

1 tests failed.
FAILED:  org.apache.solr.cloud.BasicDistributedZkTest.testDistribSearch

Error Message:
There are still nodes recoverying - waited for 230 seconds

Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 230 seconds
	at __randomizedtesting.SeedInfo.seed([2BB5D8C3FA8C2A9E:AA5356DB8DD34AA2]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:168)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:126)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:121)
	at org.apache.solr.cloud.BasicDistributedZkTest.testCoreUnloadAndLeaders(BasicDistributedZkTest.java:586)
	at org.apache.solr.cloud.BasicDistributedZkTest.doTest(BasicDistributedZkTest.java:337)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:794)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:722)




Build Log:
[...truncated 9330 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
[junit4:junit4]   2> 0 T1077 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /hh/ba
[junit4:junit4]   2> 6 T1077 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZkTest-1359698759686
[junit4:junit4]   2> 7 T1077 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 8 T1078 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 10 T1078 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 10 T1078 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4]   2> 10 T1078 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4]   2> 10 T1078 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 11 T1078 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4]   2> 109 T1077 oasc.ZkTestServer.run start zk server on port:22847
[junit4:junit4]   2> 109 T1077 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:22847 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@660a8051
[junit4:junit4]   2> 110 T1077 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 110 T1083 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:22847. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 111 T1083 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:22847, initiating session
[junit4:junit4]   2> 111 T1079 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:22848
[junit4:junit4]   2> 112 T1079 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:22848
[junit4:junit4]   2> 112 T1081 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 116 T1081 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c945c900b0000 with negotiated timeout 10000 for client /140.211.11.196:22848
[junit4:junit4]   2> 116 T1083 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:22847, sessionid = 0x13c945c900b0000, negotiated timeout = 10000
[junit4:junit4]   2> 116 T1084 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@660a8051 name:ZooKeeperConnection Watcher:127.0.0.1:22847 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 116 T1077 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 117 T1077 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 120 T1082 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c945c900b0000
[junit4:junit4]   2> 121 T1084 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 121 T1077 oaz.ZooKeeper.close Session: 0x13c945c900b0000 closed
[junit4:junit4]   2> 121 T1079 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:22848 which had sessionid 0x13c945c900b0000
[junit4:junit4]   2> 122 T1077 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:22847/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@403ba71e
[junit4:junit4]   2> 122 T1077 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 123 T1085 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:22847. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 123 T1085 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:22847, initiating session
[junit4:junit4]   2> 123 T1079 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:22849
[junit4:junit4]   2> 124 T1079 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:22849
[junit4:junit4]   2> 130 T1081 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c945c900b0001 with negotiated timeout 10000 for client /140.211.11.196:22849
[junit4:junit4]   2> 130 T1085 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:22847, sessionid = 0x13c945c900b0001, negotiated timeout = 10000
[junit4:junit4]   2> 130 T1086 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@403ba71e name:ZooKeeperConnection Watcher:127.0.0.1:22847/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 130 T1077 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 131 T1077 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 135 T1077 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 144 T1077 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 161 T1077 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 165 T1077 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 166 T1077 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 178 T1077 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 179 T1077 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 183 T1077 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 184 T1077 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 187 T1077 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 188 T1077 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 195 T1077 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 196 T1077 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 199 T1077 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 200 T1077 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 203 T1077 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 204 T1077 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 207 T1077 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 207 T1077 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 211 T1077 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 211 T1077 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 214 T1082 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c945c900b0001
[junit4:junit4]   2> 221 T1086 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 221 T1079 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:22849 which had sessionid 0x13c945c900b0001
[junit4:junit4]   2> 221 T1077 oaz.ZooKeeper.close Session: 0x13c945c900b0001 closed
[junit4:junit4]   2> 390 T1077 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 396 T1077 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:22850
[junit4:junit4]   2> 396 T1077 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 397 T1077 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 397 T1077 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359698759901
[junit4:junit4]   2> 398 T1077 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359698759901/solr.xml
[junit4:junit4]   2> 398 T1077 oasc.CoreContainer.<init> New CoreContainer 1629484333
[junit4:junit4]   2> 399 T1077 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359698759901/'
[junit4:junit4]   2> 399 T1077 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359698759901/'
[junit4:junit4]   2> 454 T1077 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 455 T1077 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 455 T1077 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 456 T1077 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 456 T1077 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 457 T1077 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 457 T1077 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 457 T1077 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 458 T1077 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 458 T1077 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 482 T1077 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 501 T1077 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:22847/solr
[junit4:junit4]   2> 501 T1077 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 502 T1077 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:22847 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@696b1d46
[junit4:junit4]   2> 503 T1077 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 503 T1096 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:22847. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 504 T1096 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:22847, initiating session
[junit4:junit4]   2> 505 T1079 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:22851
[junit4:junit4]   2> 505 T1079 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:22851
[junit4:junit4]   2> 511 T1081 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c945c900b0002 with negotiated timeout 20000 for client /140.211.11.196:22851
[junit4:junit4]   2> 511 T1096 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:22847, sessionid = 0x13c945c900b0002, negotiated timeout = 20000
[junit4:junit4]   2> 511 T1097 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@696b1d46 name:ZooKeeperConnection Watcher:127.0.0.1:22847 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 511 T1077 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 513 T1082 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c945c900b0002
[junit4:junit4]   2> 514 T1097 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 514 T1079 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:22851 which had sessionid 0x13c945c900b0002
[junit4:junit4]   2> 514 T1077 oaz.ZooKeeper.close Session: 0x13c945c900b0002 closed
[junit4:junit4]   2> 515 T1077 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 528 T1077 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:22847/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@5e5499e4
[junit4:junit4]   2> 529 T1077 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 529 T1098 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:22847. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 530 T1098 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:22847, initiating session
[junit4:junit4]   2> 530 T1079 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:22852
[junit4:junit4]   2> 530 T1079 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:22852
[junit4:junit4]   2> 531 T1081 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c945c900b0003 with negotiated timeout 20000 for client /140.211.11.196:22852
[junit4:junit4]   2> 532 T1098 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:22847, sessionid = 0x13c945c900b0003, negotiated timeout = 20000
[junit4:junit4]   2> 532 T1099 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5e5499e4 name:ZooKeeperConnection Watcher:127.0.0.1:22847/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 532 T1077 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 534 T1082 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c945c900b0003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 543 T1077 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 546 T1077 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:22850_hh%2Fba
[junit4:junit4]   2> 546 T1082 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c945c900b0003 type:delete cxid:0x7 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:22850_hh%2Fba Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:22850_hh%2Fba
[junit4:junit4]   2> 548 T1077 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:22850_hh%2Fba
[junit4:junit4]   2> 557 T1077 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 571 T1082 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c945c900b0003 type:delete cxid:0x15 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 572 T1077 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 575 T1077 oasc.Overseer.start Overseer (id=89109217915109379-127.0.0.1:22850_hh%2Fba-n_0000000000) starting
[junit4:junit4]   2> 576 T1082 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c945c900b0003 type:create cxid:0x1a zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 583 T1082 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c945c900b0003 type:create cxid:0x1b zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 584 T1082 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c945c900b0003 type:create cxid:0x1c zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 591 T1082 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c945c900b0003 type:create cxid:0x1d zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 592 T1101 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 593 T1077 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 604 T1077 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 606 T1100 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 611 T1102 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359698759901/collection1
[junit4:junit4]   2> 612 T1102 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 612 T1102 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 612 T1102 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 614 T1102 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359698759901/collection1/'
[junit4:junit4]   2> 616 T1102 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359698759901/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 616 T1102 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359698759901/collection1/lib/README' to classloader
[junit4:junit4]   2> 669 T1102 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 745 T1102 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 747 T1102 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 754 T1102 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 1383 T1102 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1395 T1102 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1399 T1102 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1417 T1102 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1422 T1102 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1427 T1102 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1428 T1102 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1429 T1102 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359698759901/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/control/data/
[junit4:junit4]   2> 1430 T1102 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@34c3c29
[junit4:junit4]   2> 1430 T1102 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 1431 T1102 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/control/data forceNew:false
[junit4:junit4]   2> 1432 T1102 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/control/data
[junit4:junit4]   2> 1432 T1102 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/control/data/index/
[junit4:junit4]   2> 1433 T1102 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1434 T1102 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/control/data/index forceNew:false
[junit4:junit4]   2> 1439 T1102 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6e604bcd),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 1440 T1102 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 1441 T1102 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/control/data/index
[junit4:junit4]   2> 1441 T1102 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 1443 T1102 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1443 T1102 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 1443 T1102 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1444 T1102 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 1444 T1102 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1445 T1102 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1445 T1102 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1445 T1102 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1446 T1102 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1446 T1102 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 1447 T1102 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 1447 T1102 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1447 T1102 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 1448 T1102 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 1448 T1102 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 1448 T1102 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1449 T1102 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1449 T1102 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1450 T1102 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1450 T1102 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1451 T1102 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1451 T1102 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1451 T1102 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1452 T1102 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1452 T1102 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 1453 T1102 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 1464 T1102 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1469 T1102 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 1470 T1102 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 1472 T1102 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/control/data
[junit4:junit4]   2> 1474 T1102 oass.SolrIndexSearcher.<init> Opening Searcher@56599200 main
[junit4:junit4]   2> 1475 T1102 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1475 T1102 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1476 T1102 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 1494 T1102 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 1549 T1103 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@56599200 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1553 T1102 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 1554 T1102 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 1554 T1082 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c945c900b0003 type:create cxid:0x50 zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 2111 T1100 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2112 T1100 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:22850_hh%2Fba",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:22850/hh/ba"}
[junit4:junit4]   2> 2112 T1100 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4]   2> 2112 T1100 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 2113 T1082 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c945c900b0003 type:create cxid:0x58 zxid:0x2c txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 2135 T1099 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 2565 T1102 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 2565 T1102 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:22850/hh/ba collection:control_collection shard:shard1
[junit4:junit4]   2> 2566 T1102 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4]   2> 2582 T1082 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c945c900b0003 type:delete cxid:0x72 zxid:0x36 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4]   2> 2583 T1102 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 2584 T1082 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c945c900b0003 type:create cxid:0x73 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2594 T1102 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 2595 T1102 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 2595 T1102 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:22850/hh/ba/collection1/
[junit4:junit4]   2> 2595 T1102 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 2595 T1102 oasc.SyncStrategy.syncToMe http://127.0.0.1:22850/hh/ba/collection1/ has no replicas
[junit4:junit4]   2> 2596 T1102 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:22850/hh/ba/collection1/
[junit4:junit4]   2> 2596 T1102 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4]   2> 2601 T1082 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c945c900b0003 type:create cxid:0x7d zxid:0x3b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3640 T1100 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3664 T1099 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 3726 T1102 oasc.ZkController.register We are http://127.0.0.1:22850/hh/ba/collection1/ and leader is http://127.0.0.1:22850/hh/ba/collection1/
[junit4:junit4]   2> 3726 T1102 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:22850/hh/ba
[junit4:junit4]   2> 3726 T1102 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 3726 T1102 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 3727 T1102 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 3730 T1102 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3731 T1077 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 3731 T1077 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 3732 T1077 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 3747 T1077 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 3758 T1077 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:22847/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@1bf2f5b5
[junit4:junit4]   2> 3759 T1077 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 3759 T1104 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:22847. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 3760 T1104 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:22847, initiating session
[junit4:junit4]   2> 3760 T1079 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:22853
[junit4:junit4]   2> 3760 T1079 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:22853
[junit4:junit4]   2> 3762 T1081 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c945c900b0004 with negotiated timeout 10000 for client /140.211.11.196:22853
[junit4:junit4]   2> 3762 T1104 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:22847, sessionid = 0x13c945c900b0004, negotiated timeout = 10000
[junit4:junit4]   2> 3762 T1105 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1bf2f5b5 name:ZooKeeperConnection Watcher:127.0.0.1:22847/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 3762 T1077 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 3764 T1077 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 3766 T1077 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 3935 T1077 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 3938 T1077 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:22854
[junit4:junit4]   2> 3938 T1077 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 3939 T1077 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 3939 T1077 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359698763446
[junit4:junit4]   2> 3940 T1077 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359698763446/solr.xml
[junit4:junit4]   2> 3940 T1077 oasc.CoreContainer.<init> New CoreContainer 212711045
[junit4:junit4]   2> 3941 T1077 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359698763446/'
[junit4:junit4]   2> 3941 T1077 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359698763446/'
[junit4:junit4]   2> 3996 T1077 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 3997 T1077 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 3997 T1077 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 3998 T1077 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 3998 T1077 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 3999 T1077 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 3999 T1077 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 4000 T1077 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 4000 T1077 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 4000 T1077 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 4021 T1077 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 4041 T1077 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:22847/solr
[junit4:junit4]   2> 4041 T1077 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 4042 T1077 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:22847 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@1de5e046
[junit4:junit4]   2> 4043 T1077 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4043 T1115 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:22847. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4044 T1115 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:22847, initiating session
[junit4:junit4]   2> 4044 T1079 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:22855
[junit4:junit4]   2> 4045 T1079 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:22855
[junit4:junit4]   2> 4047 T1081 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c945c900b0005 with negotiated timeout 20000 for client /140.211.11.196:22855
[junit4:junit4]   2> 4047 T1115 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:22847, sessionid = 0x13c945c900b0005, negotiated timeout = 20000
[junit4:junit4]   2> 4047 T1116 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1de5e046 name:ZooKeeperConnection Watcher:127.0.0.1:22847 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4047 T1077 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4049 T1082 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c945c900b0005
[junit4:junit4]   2> 4058 T1116 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 4058 T1079 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:22855 which had sessionid 0x13c945c900b0005
[junit4:junit4]   2> 4058 T1077 oaz.ZooKeeper.close Session: 0x13c945c900b0005 closed
[junit4:junit4]   2> 4059 T1077 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 4070 T1077 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:22847/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@7017a134
[junit4:junit4]   2> 4071 T1077 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4072 T1117 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:22847. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4072 T1117 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:22847, initiating session
[junit4:junit4]   2> 4073 T1079 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:22856
[junit4:junit4]   2> 4073 T1079 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:22856
[junit4:junit4]   2> 4074 T1081 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c945c900b0006 with negotiated timeout 20000 for client /140.211.11.196:22856
[junit4:junit4]   2> 4074 T1117 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:22847, sessionid = 0x13c945c900b0006, negotiated timeout = 20000
[junit4:junit4]   2> 4075 T1118 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7017a134 name:ZooKeeperConnection Watcher:127.0.0.1:22847/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4075 T1077 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4076 T1082 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c945c900b0006 type:create cxid:0x1 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4078 T1082 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c945c900b0006 type:create cxid:0x2 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4086 T1077 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:22854_hh%2Fba
[junit4:junit4]   2> 4086 T1082 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c945c900b0006 type:delete cxid:0x4 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:22854_hh%2Fba Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:22854_hh%2Fba
[junit4:junit4]   2> 4087 T1077 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:22854_hh%2Fba
[junit4:junit4]   2> 4090 T1099 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 4091 T1105 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 4092 T1099 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 4100 T1077 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 4107 T1119 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359698763446/collection1
[junit4:junit4]   2> 4107 T1119 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 4108 T1119 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 4108 T1119 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 4110 T1119 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359698763446/collection1/'
[junit4:junit4]   2> 4111 T1119 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359698763446/collection1/lib/README' to classloader
[junit4:junit4]   2> 4111 T1119 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359698763446/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 4163 T1119 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 4232 T1119 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 4234 T1119 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 4241 T1119 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 4856 T1119 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 4868 T1119 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 4872 T1119 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 4891 T1119 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 4896 T1119 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 4901 T1119 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 4902 T1119 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 4904 T1119 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359698763446/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty1/
[junit4:junit4]   2> 4904 T1119 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@34c3c29
[junit4:junit4]   2> 4905 T1119 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 4906 T1119 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty1 forceNew:false
[junit4:junit4]   2> 4906 T1119 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty1
[junit4:junit4]   2> 4907 T1119 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty1/index/
[junit4:junit4]   2> 4907 T1119 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 4908 T1119 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty1/index forceNew:false
[junit4:junit4]   2> 4913 T1119 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@20bb94f4),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 4914 T1119 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 4914 T1119 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty1/index
[junit4:junit4]   2> 4915 T1119 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 4917 T1119 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 4917 T1119 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 4917 T1119 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 4918 T1119 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 4918 T1119 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 4919 T1119 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 4919 T1119 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 4920 T1119 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 4920 T1119 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 4921 T1119 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 4921 T1119 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 4922 T1119 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 4922 T1119 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 4922 T1119 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 4923 T1119 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 4923 T1119 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 4924 T1119 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4924 T1119 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4925 T1119 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4925 T1119 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4926 T1119 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4926 T1119 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4927 T1119 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4927 T1119 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 4928 T1119 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 4928 T1119 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 4940 T1119 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 4944 T1119 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 4946 T1119 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 4948 T1119 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty1
[junit4:junit4]   2> 4950 T1119 oass.SolrIndexSearcher.<init> Opening Searcher@533cd694 main
[junit4:junit4]   2> 4951 T1119 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 4951 T1119 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 4951 T1119 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 4958 T1119 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 5008 T1120 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@533cd694 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 5013 T1119 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 5013 T1119 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 5178 T1100 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 5179 T1100 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:22850_hh%2Fba",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:22850/hh/ba"}
[junit4:junit4]   2> 5185 T1100 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:22854_hh%2Fba",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:22854/hh/ba"}
[junit4:junit4]   2> 5185 T1100 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4]   2> 5186 T1100 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 5201 T1099 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5201 T1105 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5201 T1118 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6016 T1119 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 6016 T1119 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:22854/hh/ba collection:collection1 shard:shard1
[junit4:junit4]   2> 6017 T1119 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 6035 T1082 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c945c900b0006 type:delete cxid:0x43 zxid:0x5a txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 6043 T1119 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 6044 T1082 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c945c900b0006 type:create cxid:0x44 zxid:0x5b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6058 T1119 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 6058 T1119 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 6058 T1119 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:22854/hh/ba/collection1/
[junit4:junit4]   2> 6059 T1119 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 6059 T1119 oasc.SyncStrategy.syncToMe http://127.0.0.1:22854/hh/ba/collection1/ has no replicas
[junit4:junit4]   2> 6059 T1119 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:22854/hh/ba/collection1/
[junit4:junit4]   2> 6059 T1119 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 6078 T1082 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c945c900b0006 type:create cxid:0x4e zxid:0x5f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 6719 T1100 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6742 T1099 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6742 T1105 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6742 T1118 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6795 T1119 oasc.ZkController.register We are http://127.0.0.1:22854/hh/ba/collection1/ and leader is http://127.0.0.1:22854/hh/ba/collection1/
[junit4:junit4]   2> 6795 T1119 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:22854/hh/ba
[junit4:junit4]   2> 6795 T1119 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 6795 T1119 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 6796 T1119 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 6799 T1119 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6800 T1077 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 6801 T1077 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 6801 T1077 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 6963 T1077 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 6966 T1077 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:22862
[junit4:junit4]   2> 6966 T1077 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 6967 T1077 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 6967 T1077 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359698766496
[junit4:junit4]   2> 6967 T1077 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359698766496/solr.xml
[junit4:junit4]   2> 6968 T1077 oasc.CoreContainer.<init> New CoreContainer 532548638
[junit4:junit4]   2> 6968 T1077 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359698766496/'
[junit4:junit4]   2> 6969 T1077 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359698766496/'
[junit4:junit4]   2> 7006 T1077 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 7007 T1077 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 7007 T1077 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 7008 T1077 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 7008 T1077 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 7008 T1077 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 7009 T1077 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 7009 T1077 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 7009 T1077 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 7009 T1077 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 7024 T1077 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 7038 T1077 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:22847/solr
[junit4:junit4]   2> 7039 T1077 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 7039 T1077 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:22847 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@4a78fa48
[junit4:junit4]   2> 7040 T1077 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 7040 T1130 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:22847. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 7041 T1130 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:22847, initiating session
[junit4:junit4]   2> 7041 T1079 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:22863
[junit4:junit4]   2> 7042 T1079 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:22863
[junit4:junit4]   2> 7044 T1081 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c945c900b0007 with negotiated timeout 20000 for client /140.211.11.196:22863
[junit4:junit4]   2> 7044 T1130 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:22847, sessionid = 0x13c945c900b0007, negotiated timeout = 20000
[junit4:junit4]   2> 7044 T1131 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4a78fa48 name:ZooKeeperConnection Watcher:127.0.0.1:22847 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 7044 T1077 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 7046 T1082 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c945c900b0007
[junit4:junit4]   2> 7055 T1131 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 7055 T1079 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:22863 which had sessionid 0x13c945c900b0007
[junit4:junit4]   2> 7055 T1077 oaz.ZooKeeper.close Session: 0x13c945c900b0007 closed
[junit4:junit4]   2> 7056 T1077 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 7064 T1077 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:22847/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@907aad0
[junit4:junit4]   2> 7065 T1077 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 7065 T1132 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:22847. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 7066 T1132 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:22847, initiating session
[junit4:junit4]   2> 7066 T1079 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:22864
[junit4:junit4]   2> 7066 T1079 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:22864
[junit4:junit4]   2> 7072 T1081 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c945c900b0008 with negotiated timeout 20000 for client /140.211.11.196:22864
[junit4:junit4]   2> 7072 T1132 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:22847, sessionid = 0x13c945c900b0008, negotiated timeout = 20000
[junit4:junit4]   2> 7072 T1133 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@907aad0 name:ZooKeeperConnection Watcher:127.0.0.1:22847/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 7072 T1077 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 7073 T1082 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c945c900b0008 type:create cxid:0x1 zxid:0x6c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7075 T1082 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c945c900b0008 type:create cxid:0x2 zxid:0x6d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7084 T1077 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:22862_hh%2Fba
[junit4:junit4]   2> 7085 T1082 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c945c900b0008 type:delete cxid:0x4 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:22862_hh%2Fba Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:22862_hh%2Fba
[junit4:junit4]   2> 7091 T1077 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:22862_hh%2Fba
[junit4:junit4]   2> 7094 T1105 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7095 T1099 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7095 T1118 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7095 T1118 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 7097 T1105 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7105 T1099 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 7106 T1077 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 7112 T1134 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359698766496/collection1
[junit4:junit4]   2> 7112 T1134 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 7113 T1134 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 7113 T1134 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 7115 T1134 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359698766496/collection1/'
[junit4:junit4]   2> 7116 T1134 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359698766496/collection1/lib/README' to classloader
[junit4:junit4]   2> 7117 T1134 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359698766496/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 7173 T1134 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 7242 T1134 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 7243 T1134 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 7251 T1134 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 7868 T1134 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 7880 T1134 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 7884 T1134 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 7902 T1134 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 7907 T1134 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 7912 T1134 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 7913 T1134 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 7915 T1134 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359698766496/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty2/
[junit4:junit4]   2> 7915 T1134 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@34c3c29
[junit4:junit4]   2> 7915 T1134 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 7916 T1134 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty2 forceNew:false
[junit4:junit4]   2> 7917 T1134 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty2
[junit4:junit4]   2> 7917 T1134 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty2/index/
[junit4:junit4]   2> 7917 T1134 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 7918 T1134 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty2/index forceNew:false
[junit4:junit4]   2> 7923 T1134 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@66e0f67),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 7923 T1134 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 7924 T1134 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty2/index
[junit4:junit4]   2> 7924 T1134 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 7926 T1134 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 7927 T1134 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 7927 T1134 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 7927 T1134 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 7928 T1134 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 7928 T1134 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 7929 T1134 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 7929 T1134 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 7930 T1134 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 7930 T1134 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 7931 T1134 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 7931 T1134 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 7932 T1134 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 7932 T1134 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 7932 T1134 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 7933 T1134 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 7933 T1134 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7934 T1134 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7934 T1134 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7935 T1134 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7935 T1134 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7936 T1134 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7936 T1134 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7937 T1134 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 7937 T1134 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 7938 T1134 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 7949 T1134 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 7954 T1134 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 7955 T1134 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 7957 T1134 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty2
[junit4:junit4]   2> 7959 T1134 oass.SolrIndexSearcher.<init> Opening Searcher@6d9db843 main
[junit4:junit4]   2> 7960 T1134 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 7960 T1134 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 7960 T1134 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 7967 T1134 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 8023 T1135 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6d9db843 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 8029 T1134 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 8029 T1134 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 8255 T1100 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8256 T1100 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:22854_hh%2Fba",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:22854/hh/ba"}
[junit4:junit4]   2> 8261 T1100 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:22862_hh%2Fba",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:22862/hh/ba"}
[junit4:junit4]   2> 8261 T1100 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 8261 T1100 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 8272 T1099 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8272 T1105 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8272 T1118 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8272 T1133 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9031 T1134 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 9031 T1134 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:22862/hh/ba collection:collection1 shard:shard2
[junit4:junit4]   2> 9032 T1134 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4]   2> 9050 T1082 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c945c900b0008 type:delete cxid:0x42 zxid:0x7c txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
[junit4:junit4]   2> 9051 T1134 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 9052 T1082 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c945c900b0008 type:create cxid:0x43 zxid:0x7d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9054 T1134 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 9054 T1134 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 9054 T1134 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:22862/hh/ba/collection1/
[junit4:junit4]   2> 9055 T1134 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 9055 T1134 oasc.SyncStrategy.syncToMe http://127.0.0.1:22862/hh/ba/collection1/ has no replicas
[junit4:junit4]   2> 9055 T1134 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:22862/hh/ba/collection1/
[junit4:junit4]   2> 9055 T1134 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4]   2> 9059 T1082 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c945c900b0008 type:create cxid:0x4c zxid:0x80 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 9787 T1100 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9805 T1099 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9805 T1133 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9805 T1118 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9805 T1105 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9827 T1134 oasc.ZkController.register We are http://127.0.0.1:22862/hh/ba/collection1/ and leader is http://127.0.0.1:22862/hh/ba/collection1/
[junit4:junit4]   2> 9827 T1134 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:22862/hh/ba
[junit4:junit4]   2> 9827 T1134 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 9827 T1134 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 9828 T1134 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 9830 T1134 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9831 T1077 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 9831 T1077 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 9832 T1077 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 10021 T1077 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 10025 T1077 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:22867
[junit4:junit4]   2> 10025 T1077 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 10026 T1077 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 10026 T1077 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359698769526
[junit4:junit4]   2> 10027 T1077 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359698769526/solr.xml
[junit4:junit4]   2> 10027 T1077 oasc.CoreContainer.<init> New CoreContainer 474455243
[junit4:junit4]   2> 10028 T1077 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359698769526/'
[junit4:junit4]   2> 10029 T1077 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359698769526/'
[junit4:junit4]   2> 10086 T1077 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 10087 T1077 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 10087 T1077 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 10087 T1077 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 10088 T1077 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 10088 T1077 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 10089 T1077 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 10089 T1077 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 10090 T1077 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 10090 T1077 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 10112 T1077 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 10131 T1077 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:22847/solr
[junit4:junit4]   2> 10132 T1077 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 10133 T1077 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:22847 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@679cafd7
[junit4:junit4]   2> 10134 T1077 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 10134 T1145 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:22847. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 10135 T1145 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:22847, initiating session
[junit4:junit4]   2> 10135 T1079 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:22868
[junit4:junit4]   2> 10136 T1079 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:22868
[junit4:junit4]   2> 10138 T1081 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c945c900b0009 with negotiated timeout 20000 for client /140.211.11.196:22868
[junit4:junit4]   2> 10138 T1145 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:22847, sessionid = 0x13c945c900b0009, negotiated timeout = 20000
[junit4:junit4]   2> 10138 T1146 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@679cafd7 name:ZooKeeperConnection Watcher:127.0.0.1:22847 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 10138 T1077 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 10140 T1082 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c945c900b0009
[junit4:junit4]   2> 10153 T1079 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:22868 which had sessionid 0x13c945c900b0009
[junit4:junit4]   2> 10153 T1146 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 10153 T1077 oaz.ZooKeeper.close Session: 0x13c945c900b0009 closed
[junit4:junit4]   2> 10154 T1077 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 10165 T1077 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:22847/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@71480cee
[junit4:junit4]   2> 10167 T1077 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 10167 T1147 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:22847. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 10167 T1147 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:22847, initiating session
[junit4:junit4]   2> 10168 T1079 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:22869
[junit4:junit4]   2> 10168 T1079 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:22869
[junit4:junit4]   2> 10169 T1081 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c945c900b000a with negotiated timeout 20000 for client /140.211.11.196:22869
[junit4:junit4]   2> 10169 T1147 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:22847, sessionid = 0x13c945c900b000a, negotiated timeout = 20000
[junit4:junit4]   2> 10170 T1148 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@71480cee name:ZooKeeperConnection Watcher:127.0.0.1:22847/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 10170 T1077 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 10171 T1082 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c945c900b000a type:create cxid:0x1 zxid:0x8d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 10173 T1082 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c945c900b000a type:create cxid:0x2 zxid:0x8e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 10180 T1077 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:22867_hh%2Fba
[junit4:junit4]   2> 10181 T1082 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c945c900b000a type:delete cxid:0x4 zxid:0x8f txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:22867_hh%2Fba Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:22867_hh%2Fba
[junit4:junit4]   2> 10182 T1077 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:22867_hh%2Fba
[junit4:junit4]   2> 10185 T1099 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10185 T1133 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10185 T1105 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10185 T1118 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 10186 T1118 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 10187 T1133 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 10188 T1099 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 10188 T1105 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 10190 T1077 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 10197 T1149 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359698769526/collection1
[junit4:junit4]   2> 10198 T1149 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 10198 T1149 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 10198 T1149 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 10200 T1149 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359698769526/collection1/'
[junit4:junit4]   2> 10201 T1149 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359698769526/collection1/lib/README' to classloader
[junit4:junit4]   2> 10202 T1149 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359698769526/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 10257 T1149 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 10330 T1149 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 10331 T1149 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 10338 T1149 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 11069 T1149 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 11082 T1149 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 11086 T1149 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 11102 T1149 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 11106 T1149 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 11110 T1149 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 11111 T1149 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 11112 T1149 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359698769526/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty3/
[junit4:junit4]   2> 11112 T1149 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@34c3c29
[junit4:junit4]   2> 11113 T1149 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 11113 T1149 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty3 forceNew:false
[junit4:junit4]   2> 11114 T1149 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty3
[junit4:junit4]   2> 11114 T1149 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty3/index/
[junit4:junit4]   2> 11114 T1149 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 11115 T1149 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty3/index forceNew:false
[junit4:junit4]   2> 11122 T1149 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@76d44e69),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 11123 T1149 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 11123 T1149 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty3/index
[junit4:junit4]   2> 11123 T1149 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 11125 T1149 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 11125 T1149 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 11125 T1149 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 11126 T1149 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 11126 T1149 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 11126 T1149 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 11127 T1149 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 11127 T1149 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 11127 T1149 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 11128 T1149 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 11128 T1149 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 11129 T1149 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 11129 T1149 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 11129 T1149 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 11129 T1149 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 11130 T1149 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 11130 T1149 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11130 T1149 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11131 T1149 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11131 T1149 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11132 T1149 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11132 T1149 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11132 T1149 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11133 T1149 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 11133 T1149 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 11133 T1149 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 11142 T1149 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 11146 T1149 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 11147 T1149 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 11149 T1149 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty3
[junit4:junit4]   2> 11150 T1149 oass.SolrIndexSearcher.<init> Opening Searcher@afa4886 main
[junit4:junit4]   2> 11151 T1149 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 11151 T1149 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 11152 T1149 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 11167 T1149 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 11219 T1150 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@afa4886 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 11225 T1149 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 11225 T1149 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 11319 T1100 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11320 T1100 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:22862_hh%2Fba",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:22862/hh/ba"}
[junit4:junit4]   2> 11325 T1100 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:22867_hh%2Fba",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:22867/hh/ba"}
[junit4:junit4]   2> 11325 T1100 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 11325 T1100 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 11337 T1099 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11337 T1105 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11337 T1118 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11337 T1148 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 11337 T1133 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12228 T1149 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 12228 T1149 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:22867/hh/ba collection:collection1 shard:shard1
[junit4:junit4]   2> 12257 T1149 oasc.ZkController.register We are http://127.0.0.1:22867/hh/ba/collection1/ and leader is http://127.0.0.1:22854/hh/ba/collection1/
[junit4:junit4]   2> 12258 T1149 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:22867/hh/ba
[junit4:junit4]   2> 12258 T1149 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 12258 T1149 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C49 name=collection1 org.apache.solr.core.SolrCore@fb7034d url=http://127.0.0.1:22867/hh/ba/collection1 node=127.0.0.1:22867_hh%2Fba C49_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:22867_hh%2Fba, base_url=http://127.0.0.1:22867/hh/ba}
[junit4:junit4]   2> 12258 T1151 C49 P22867 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 12259 T1149 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12259 T1151 C49 P22867 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 12260 T1151 C49 P22867 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 12260 T1151 C49 P22867 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 12260 T1077 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 12261 T1077 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 12261 T1151 C49 P22867 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 12261 T1077 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 12451 T1077 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 12454 T1077 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:22875
[junit4:junit4]   2> 12455 T1077 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 12455 T1077 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 12456 T1077 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359698771958
[junit4:junit4]   2> 12456 T1077 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359698771958/solr.xml
[junit4:junit4]   2> 12456 T1077 oasc.CoreContainer.<init> New CoreContainer 1327435096
[junit4:junit4]   2> 12457 T1077 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359698771958/'
[junit4:junit4]   2> 12458 T1077 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359698771958/'
[junit4:junit4]   2> 12510 T1077 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 12511 T1077 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 12512 T1077 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 12512 T1077 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 12512 T1077 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 12513 T1077 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 12513 T1077 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 12514 T1077 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 12514 T1077 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 12515 T1077 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 12537 T1077 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 12557 T1077 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:22847/solr
[junit4:junit4]   2> 12558 T1077 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 12558 T1077 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:22847 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@46af2832
[junit4:junit4]   2> 12559 T1077 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 12560 T1161 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:22847. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 12561 T1161 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:22847, initiating session
[junit4:junit4]   2> 12561 T1079 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:22876
[junit4:junit4]   2> 12562 T1079 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:22876
[junit4:junit4]   2> 12563 T1081 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c945c900b000b with negotiated timeout 20000 for client /140.211.11.196:22876
[junit4:junit4]   2> 12563 T1161 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:22847, sessionid = 0x13c945c900b000b, negotiated timeout = 20000
[junit4:junit4]   2> 12563 T1162 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@46af2832 name:ZooKeeperConnection Watcher:127.0.0.1:22847 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 12564 T1077 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 12565 T1082 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c945c900b000b
[junit4:junit4]   2> 12566 T1162 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 12566 T1079 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:22876 which had sessionid 0x13c945c900b000b
[junit4:junit4]   2> 12566 T1077 oaz.ZooKeeper.close Session: 0x13c945c900b000b closed
[junit4:junit4]   2> 12567 T1077 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 12578 T1077 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:22847/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@27b1e5f5
[junit4:junit4]   2> 12579 T1077 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 12579 T1163 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:22847. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 12580 T1163 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:22847, initiating session
[junit4:junit4]   2> 12580 T1079 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:22877
[junit4:junit4]   2> 12581 T1079 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:22877
[junit4:junit4]   2> 12582 T1081 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c945c900b000c with negotiated timeout 20000 for client /140.211.11.196:22877
[junit4:junit4]   2> 12582 T1163 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:22847, sessionid = 0x13c945c900b000c, negotiated timeout = 20000
[junit4:junit4]   2> 12582 T1164 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@27b1e5f5 name:ZooKeeperConnection Watcher:127.0.0.1:22847/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 12582 T1077 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 12584 T1082 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c945c900b000c type:create cxid:0x1 zxid:0x9f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12585 T1082 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c945c900b000c type:create cxid:0x2 zxid:0xa0 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12593 T1077 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:22875_hh%2Fba
[junit4:junit4]   2> 12594 T1082 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c945c900b000c type:delete cxid:0x4 zxid:0xa1 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:22875_hh%2Fba Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:22875_hh%2Fba
[junit4:junit4]   2> 12595 T1077 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:22875_hh%2Fba
[junit4:junit4]   2> 12597 T1099 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12597 T1105 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12597 T1133 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 12598 T1118 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 12598 T1118 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 12598 T1148 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 12599 T1148 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 12600 T1105 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 12600 T1099 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 12601 T1133 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 12603 T1077 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 12611 T1165 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359698771958/collection1
[junit4:junit4]   2> 12611 T1165 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 12612 T1165 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 12612 T1165 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 12614 T1165 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359698771958/collection1/'
[junit4:junit4]   2> 12615 T1165 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359698771958/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 12616 T1165 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359698771958/collection1/lib/README' to classloader
[junit4:junit4]   2> 12669 T1165 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 12743 T1165 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 12744 T1165 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 12751 T1165 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 12852 T1100 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 12853 T1100 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:22867_hh%2Fba",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:22867/hh/ba"}
[junit4:junit4]   2> 12867 T1164 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 12867 T1148 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 12867 T1118 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 12867 T1099 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 12867 T1105 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 12867 T1133 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 13422 T1165 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 13431 T1165 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 13434 T1165 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 13448 T1165 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 13452 T1165 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 13456 T1165 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 13457 T1165 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 13458 T1165 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359698771958/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty4/
[junit4:junit4]   2> 13458 T1165 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@34c3c29
[junit4:junit4]   2> 13459 T1165 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 13459 T1165 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty4 forceNew:false
[junit4:junit4]   2> 13460 T1165 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty4
[junit4:junit4]   2> 13460 T1165 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty4/index/
[junit4:junit4]   2> 13460 T1165 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 13461 T1165 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty4/index forceNew:false
[junit4:junit4]   2> 13494 T1165 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@20d05aab),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 13495 T1165 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 13495 T1165 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty4/index
[junit4:junit4]   2> 13495 T1165 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 13497 T1165 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 13497 T1165 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 13497 T1165 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 13498 T1165 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 13498 T1165 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 13498 T1165 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 13499 T1165 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 13499 T1165 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 13499 T1165 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 13500 T1165 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 13500 T1165 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 13500 T1165 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 13501 T1165 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 13501 T1165 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 13501 T1165 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 13502 T1165 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 13502 T1165 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13502 T1165 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13503 T1165 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13503 T1165 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13503 T1165 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13504 T1165 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13504 T1165 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13505 T1165 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 13505 T1165 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 13505 T1165 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 13513 T1165 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 13516 T1165 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 13518 T1165 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 13519 T1165 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty4
[junit4:junit4]   2> 13521 T1165 oass.SolrIndexSearcher.<init> Opening Searcher@56619a1c main
[junit4:junit4]   2> 13522 T1165 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 13522 T1165 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 13522 T1165 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 13531 T1165 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 13610 T1166 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@56619a1c main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 13614 T1165 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 13614 T1165 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 14374 T1100 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14375 T1100 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:22875_hh%2Fba",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:22875/hh/ba"}
[junit4:junit4]   2> 14375 T1100 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 14375 T1100 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 14379 T1099 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14379 T1164 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14379 T1133 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14379 T1105 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14379 T1118 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14379 T1148 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 14616 T1165 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 14616 T1165 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:22875/hh/ba collection:collection1 shard:shard2
[junit4:junit4]   2> 14620 T1165 oasc.ZkController.register We are http://127.0.0.1:22875/hh/ba/collection1/ and leader is http://127.0.0.1:22862/hh/ba/collection1/
[junit4:junit4]   2> 14621 T1165 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:22875/hh/ba
[junit4:junit4]   2> 14621 T1165 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 14621 T1165 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C50 name=collection1 org.apache.solr.core.SolrCore@76a3730d url=http://127.0.0.1:22875/hh/ba/collection1 node=127.0.0.1:22875_hh%2Fba C50_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:22875_hh%2Fba, base_url=http://127.0.0.1:22875/hh/ba}
[junit4:junit4]   2> 14622 T1167 C50 P22875 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 14622 T1167 C50 P22875 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 14623 T1165 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14623 T1167 C50 P22875 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 14623 T1167 C50 P22875 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 14624 T1077 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 14625 T1167 C50 P22875 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 14625 T1077 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 14626 T1077 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 14649 T1077 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14651 T1077 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4]   2> 14652 T1077 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C49_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:22867_hh%2Fba, base_url=http://127.0.0.1:22867/hh/ba}
[junit4:junit4]   2> 15283 T1151 C49 P22867 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:22854/hh/ba/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 15283 T1151 C49 P22867 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:22867/hh/ba START replicas=[http://127.0.0.1:22854/hh/ba/collection1/] nUpdates=100
[junit4:junit4]   2> 15283 T1151 C49 P22867 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 15284 T1151 C49 P22867 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 15284 T1151 C49 P22867 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 15284 T1151 C49 P22867 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 15285 T1151 C49 P22867 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 15285 T1151 C49 P22867 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:22854/hh/ba/collection1/. core=collection1
[junit4:junit4]   2> 15285 T1151 C49 P22867 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C51 name=collection1 org.apache.solr.core.SolrCore@3535d884 url=http://127.0.0.1:22854/hh/ba/collection1 node=127.0.0.1:22854_hh%2Fba C51_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:22854_hh%2Fba, base_url=http://127.0.0.1:22854/hh/ba, leader=true}
[junit4:junit4]   2> 15295 T1109 C51 P22854 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 15299 T1110 C51 P22854 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 15300 T1110 C51 P22854 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty1
[junit4:junit4]   2> 15304 T1110 C51 P22854 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@20bb94f4),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 15305 T1110 C51 P22854 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 15335 T1110 C51 P22854 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@20bb94f4),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@20bb94f4),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 15336 T1110 C51 P22854 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 15337 T1110 C51 P22854 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty1
[junit4:junit4]   2> 15337 T1110 C51 P22854 oass.SolrIndexSearcher.<init> Opening Searcher@59347307 realtime
[junit4:junit4]   2> 15338 T1110 C51 P22854 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15338 T1110 C51 P22854 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 39
[junit4:junit4]   2> 15339 T1151 C49 P22867 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 15339 T1151 C49 P22867 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 15342 T1111 C51 P22854 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 15342 T1111 C51 P22854 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2 
[junit4:junit4]   2> 15343 T1151 C49 P22867 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 15343 T1151 C49 P22867 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 15343 T1151 C49 P22867 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 15345 T1111 C51 P22854 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty1
[junit4:junit4]   2> 15346 T1111 C51 P22854 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty1/index
[junit4:junit4]   2> 15346 T1111 C51 P22854 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15347 T1151 C49 P22867 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 15348 T1151 C49 P22867 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty3/index.20130201120615026 forceNew:false
[junit4:junit4]   2> 15349 T1151 C49 P22867 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty3
[junit4:junit4]   2> 15349 T1151 C49 P22867 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty3/index.20130201120615026 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b6fc63e) fullCopy=true
[junit4:junit4]   2> 15353 T1111 C51 P22854 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> 15355 T1151 C49 P22867 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty3/index
[junit4:junit4]   2> 15355 T1151 C49 P22867 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 15355 T1151 C49 P22867 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130201120615026
[junit4:junit4]   2> 15357 T1151 C49 P22867 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty3
[junit4:junit4]   2> 15357 T1151 C49 P22867 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty3
[junit4:junit4]   2> 15357 T1151 C49 P22867 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty3/index/ new=./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty3/index.20130201120615026
[junit4:junit4]   2> 15361 T1151 C49 P22867 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty3/index.20130201120615026 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b6fc63e),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 15361 T1151 C49 P22867 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 15363 T1151 C49 P22867 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty3
[junit4:junit4]   2> 15363 T1151 C49 P22867 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 15363 T1151 C49 P22867 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 15363 T1151 C49 P22867 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 15364 T1151 C49 P22867 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty3/index.20130201120615026
[junit4:junit4]   2> 15365 T1151 C49 P22867 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty3
[junit4:junit4]   2> 15368 T1151 C49 P22867 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty3/index.20130201120615026 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b6fc63e),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 15368 T1151 C49 P22867 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 15368 T1151 C49 P22867 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 15369 T1151 C49 P22867 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty3
[junit4:junit4]   2> 15369 T1151 C49 P22867 oass.SolrIndexSearcher.<init> Opening Searcher@1894199e main
[junit4:junit4]   2> 15371 T1150 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1894199e main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 15371 T1150 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty3/index
[junit4:junit4]   2> 15372 T1151 C49 P22867 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 15374 T1151 C49 P22867 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty3/index.20130201120615026 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b6fc63e),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty3/index.20130201120615026 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6b6fc63e),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 15374 T1151 C49 P22867 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 15375 T1151 C49 P22867 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty3
[junit4:junit4]   2> 15375 T1151 C49 P22867 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15376 T1151 C49 P22867 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty3/index.20130201120615026
[junit4:junit4]   2> 15376 T1151 C49 P22867 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty3/index
[junit4:junit4]   2> 15376 T1151 C49 P22867 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 15376 T1151 C49 P22867 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 15376 T1151 C49 P22867 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 15377 T1151 C49 P22867 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 15384 T1151 C49 P22867 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 15654 T1077 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 15885 T1100 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 15886 T1100 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:22875_hh%2Fba",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:22875/hh/ba"}
[junit4:junit4]   2> 15944 T1100 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:22867_hh%2Fba",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:22867/hh/ba"}
[junit4:junit4]   2> 15979 T1099 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15979 T1164 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15979 T1133 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15979 T1105 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15979 T1148 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15979 T1118 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16657 T1077 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17660 T1077 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C50_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:22875_hh%2Fba, base_url=http://127.0.0.1:22875/hh/ba}
[junit4:junit4]   2> 18650 T1167 C50 P22875 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:22862/hh/ba/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 18650 T1167 C50 P22875 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:22875/hh/ba START replicas=[http://127.0.0.1:22862/hh/ba/collection1/] nUpdates=100
[junit4:junit4]   2> 18651 T1167 C50 P22875 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 18651 T1167 C50 P22875 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 18651 T1167 C50 P22875 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 18651 T1167 C50 P22875 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 18651 T1167 C50 P22875 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 18651 T1167 C50 P22875 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:22862/hh/ba/collection1/. core=collection1
[junit4:junit4]   2> 18652 T1167 C50 P22875 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C52 name=collection1 org.apache.solr.core.SolrCore@12ae53be url=http://127.0.0.1:22862/hh/ba/collection1 node=127.0.0.1:22862_hh%2Fba C52_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:22862_hh%2Fba, base_url=http://127.0.0.1:22862/hh/ba, leader=true}
[junit4:junit4]   2> 18662 T1124 C52 P22862 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 18663 T1077 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 18669 T1125 C52 P22862 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 18670 T1125 C52 P22862 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty2
[junit4:junit4]   2> 18674 T1125 C52 P22862 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@66e0f67),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 18675 T1125 C52 P22862 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 18678 T1125 C52 P22862 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@66e0f67),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@66e0f67),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 18678 T1125 C52 P22862 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 18680 T1125 C52 P22862 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty2
[junit4:junit4]   2> 18680 T1125 C52 P22862 oass.SolrIndexSearcher.<init> Opening Searcher@152df512 realtime
[junit4:junit4]   2> 18680 T1125 C52 P22862 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 18681 T1125 C52 P22862 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 12
[junit4:junit4]   2> 18682 T1167 C50 P22875 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 18682 T1167 C50 P22875 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 18684 T1126 C52 P22862 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 18684 T1126 C52 P22862 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 18685 T1167 C50 P22875 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 18685 T1167 C50 P22875 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 18685 T1167 C50 P22875 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 18687 T1126 C52 P22862 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty2
[junit4:junit4]   2> 18688 T1126 C52 P22862 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty2/index
[junit4:junit4]   2> 18688 T1126 C52 P22862 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1 
[junit4:junit4]   2> 18689 T1167 C50 P22875 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 18690 T1167 C50 P22875 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty4/index.20130201120618368 forceNew:false
[junit4:junit4]   2> 18691 T1167 C50 P22875 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty4
[junit4:junit4]   2> 18691 T1167 C50 P22875 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty4/index.20130201120618368 lockFactory=org.apache.lucene.store.NativeFSLockFactory@69a7fa06) fullCopy=true
[junit4:junit4]   2> 18694 T1126 C52 P22862 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> 18696 T1167 C50 P22875 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty4/index
[junit4:junit4]   2> 18696 T1167 C50 P22875 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 18696 T1167 C50 P22875 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130201120618368
[junit4:junit4]   2> 18698 T1167 C50 P22875 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty4
[junit4:junit4]   2> 18698 T1167 C50 P22875 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty4
[junit4:junit4]   2> 18698 T1167 C50 P22875 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty4/index/ new=./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty4/index.20130201120618368
[junit4:junit4]   2> 18702 T1167 C50 P22875 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty4/index.20130201120618368 lockFactory=org.apache.lucene.store.NativeFSLockFactory@69a7fa06),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 18702 T1167 C50 P22875 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 18703 T1167 C50 P22875 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty4
[junit4:junit4]   2> 18704 T1167 C50 P22875 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 18704 T1167 C50 P22875 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 18704 T1167 C50 P22875 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 18705 T1167 C50 P22875 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty4/index.20130201120618368
[junit4:junit4]   2> 18706 T1167 C50 P22875 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty4
[junit4:junit4]   2> 18709 T1167 C50 P22875 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty4/index.20130201120618368 lockFactory=org.apache.lucene.store.NativeFSLockFactory@69a7fa06),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 18709 T1167 C50 P22875 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 18709 T1167 C50 P22875 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 18710 T1167 C50 P22875 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty4
[junit4:junit4]   2> 18710 T1167 C50 P22875 oass.SolrIndexSearcher.<init> Opening Searcher@667787a6 main
[junit4:junit4]   2> 18712 T1166 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@667787a6 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 18712 T1166 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty4/index
[junit4:junit4]   2> 18712 T1167 C50 P22875 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 18715 T1167 C50 P22875 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty4/index.20130201120618368 lockFactory=org.apache.lucene.store.NativeFSLockFactory@69a7fa06),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty4/index.20130201120618368 lockFactory=org.apache.lucene.store.NativeFSLockFactory@69a7fa06),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 18715 T1167 C50 P22875 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 18716 T1167 C50 P22875 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty4
[junit4:junit4]   2> 18716 T1167 C50 P22875 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 18716 T1167 C50 P22875 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty4/index.20130201120618368
[junit4:junit4]   2> 18717 T1167 C50 P22875 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty4/index
[junit4:junit4]   2> 18717 T1167 C50 P22875 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 18717 T1167 C50 P22875 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 18717 T1167 C50 P22875 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 18717 T1167 C50 P22875 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 18725 T1167 C50 P22875 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 18990 T1100 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 18991 T1100 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:22875_hh%2Fba",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:22875/hh/ba"}
[junit4:junit4]   2> 19007 T1099 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19007 T1118 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19007 T1148 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19007 T1105 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19007 T1133 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19007 T1164 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19666 T1077 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19668 T1077 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4]   2> ASYNC  NEW_CORE C53 name=collection1 org.apache.solr.core.SolrCore@4bfd274e url=http://127.0.0.1:22850/hh/ba/collection1 node=127.0.0.1:22850_hh%2Fba C53_STATE=coll:control_collection core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:22850_hh%2Fba, base_url=http://127.0.0.1:22850/hh/ba, leader=true}
[junit4:junit4]   2> 19680 T1089 C53 P22850 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/control/data
[junit4:junit4]   2> 19685 T1089 C53 P22850 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@6e604bcd),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 19685 T1089 C53 P22850 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 19687 T1089 C53 P22850 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/control/data
[junit4:junit4]   2> 19687 T1089 C53 P22850 oass.SolrIndexSearcher.<init> Opening Searcher@f5cf657 realtime
[junit4:junit4]   2> 19688 T1089 C53 P22850 /update {wt=javabin&version=2} {deleteByQuery=*:* (-1425747507264094208)} 0 9
[junit4:junit4]   2> 19692 T1127 C52 P22862 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty2
[junit4:junit4]   2> 19697 T1112 C51 P22854 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty1
[junit4:junit4]   2>  C50_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:22875_hh%2Fba, base_url=http://127.0.0.1:22875/hh/ba}
[junit4:junit4]   2> 19705 T1154 C50 P22875 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty4
[junit4:junit4]   2> 19705 T1154 C50 P22875 /update {update.distrib=FROMLEADER&_version_=-1425747507275628544&update.from=http://127.0.0.1:22862/hh/ba/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1425747507275628544)} 0 2
[junit4:junit4]   2>  C49_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:22867_hh%2Fba, base_url=http://127.0.0.1:22867/hh/ba}
[junit4:junit4]   2> 19709 T1138 C49 P22867 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-1359698759685/jetty3
[junit4:junit4]   2> 19710 T1138 C49 P22867 /update {update.distrib=FROMLEADER&_version_=-1425747507280871424&update.from=http://127.0.0.1:22854/hh/ba/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1425747507280871424)} 0 3
[junit4:junit4]   2> 19711 T1112 C51 P22854 /update {update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1425747507280871424)} 0 16
[junit4:junit4]   2> 19712 T1127 C52 P22862 /update {wt=javabin&version=2} {deleteByQuery=*:* (-1425747507275628544)} 0 21
[junit4:junit4]   2> 19722 T1089 C53 P22850 /update {wt=javabin&version=2} {add=[1 (1425747507306037248)]} 0 4
[junit4:junit4]   2> 19734 T1138 C49 P22867 /update {distrib.from=http://127.0.0.1:22854/hh/ba/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1425747507312328704)]} 0 4
[junit4:junit4]   2> 19734 T1113 C51 P22854 /update {wt=javabin&version=2} {add=[1 (1425747507312328704)]} 0 10
[junit4:junit4]   2> 19739 T1089 C53 P22850 /update {wt=javabin&version=2} {add=[2 (1425747507325960192)]} 0 2
[junit4:junit4]   2> 19758 T1154 C50 P22875 /update {distrib.from=http://127.0.0.1:22862/hh/ba/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1425747507337494528)]} 0 4
[junit4:junit4]   2> 19758 T1128 C52 P22862 /update {distrib.from=http://127.0.0.1:22854/hh/ba/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[2 (1425747507337494528)]} 0 9
[junit4:junit4]   2> 19759 T1113 C51 P22854 /update {wt=javabin&version=2} {add=[2]} 0 17
[junit4:junit4]   2> 19764 T1089 C53 P22850 /update {wt=javabin&version=2} {add=[3 (1425747507351126016)]} 0 2
[junit4:junit4]   2> 19775 T1154 C50 P22875 /update {distrib.from=http://127.0.0.1:22862/hh/ba/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[3 (1425747507359514624)]} 0 2
[junit4:junit4]   2> 19775 T1128 C52 P22862 /update {distrib.from=http://127.0.0.1:22854/hh/ba/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[3 (1425747507359514624)]} 0 6
[junit4:junit4]   2> 19776 T1113 C51 P22854 /update {wt=javabin&version=2} {add=[3]} 0 10
[junit4:junit4]   2> 19781 T1089 C53 P22850 /update {wt=javabin&version=2} {add=[4 (1425747507368951808)]} 0 2
[junit4:junit4]   2> 19789 T1138 C49 P22867 /update {distrib.from=http://127.0.0.1:22854/hh/ba/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[4 (1425747507373146112)]} 0 2
[junit4:junit4]   2> 19790 T1113 C51 P22854 /update {wt=javabin&version=2} {add=[4 (1425747507373146112)]} 0 7
[junit4:junit4]   2> 19795 T1089 C53 P22850 /update {wt=javabin&version=2} {add=[5 (1425747507383631872)]} 0 2
[junit4:junit4]   2> 19807 T1154 C50 P22875 /update {distrib.from=http://127.0.0.1:22862/hh/ba/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[5 (1425747507392020480)]} 0 2
[junit4:junit4]   2> 19807 T1128 C52 P22862 /update {distrib.from=http://127.

[...truncated too long message...]

> 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:126)
[junit4:junit4]    > 	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:121)
[junit4:junit4]    > 	at org.apache.solr.cloud.BasicDistributedZkTest.testCoreUnloadAndLeaders(BasicDistributedZkTest.java:586)
[junit4:junit4]    > 	at org.apache.solr.cloud.BasicDistributedZkTest.doTest(BasicDistributedZkTest.java:337)
[junit4:junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:794)
[junit4:junit4]    > 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4:junit4]    > 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[junit4:junit4]    > 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4:junit4]    > 	at java.lang.reflect.Method.invoke(Method.java:601)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4:junit4]    > 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]    > 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]    > 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 532016 T1077 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4]   2> NOTE: test params are: codec=Lucene41: {n_f1=Pulsing41(freqCutoff=6 minBlockSize=38 maxBlockSize=88), foo_b=MockFixedIntBlock(blockSize=1731), cat=PostingsFormat(name=NestedPulsing), foo_d=MockVariableIntBlock(baseBlockSize=104), foo_f=MockFixedIntBlock(blockSize=1731), n_tl1=Pulsing41(freqCutoff=6 minBlockSize=38 maxBlockSize=88), n_d1=PostingsFormat(name=NestedPulsing), rnd_b=MockVariableIntBlock(baseBlockSize=104), intDefault=Pulsing41(freqCutoff=6 minBlockSize=38 maxBlockSize=88), n_td1=Pulsing41(freqCutoff=6 minBlockSize=38 maxBlockSize=88), timestamp=PostingsFormat(name=NestedPulsing), id=MockVariableIntBlock(baseBlockSize=104), range_facet_sl=MockVariableIntBlock(baseBlockSize=104), range_facet_si=PostingsFormat(name=NestedPulsing), oddField_s=MockVariableIntBlock(baseBlockSize=104), sequence_i=MockVariableIntBlock(baseBlockSize=104), name=MockVariableIntBlock(baseBlockSize=104), foo_i=Pulsing41(freqCutoff=6 minBlockSize=38 maxBlockSize=88), regex_dup_B_s=PostingsFormat(name=NestedPulsing), multiDefault=PostingsFormat(name=NestedPulsing), n_tf1=PostingsFormat(name=NestedPulsing), n_dt1=Pulsing41(freqCutoff=6 minBlockSize=38 maxBlockSize=88), genre_s=MockVariableIntBlock(baseBlockSize=104), author_t=Pulsing41(freqCutoff=6 minBlockSize=38 maxBlockSize=88), n_ti1=MockVariableIntBlock(baseBlockSize=104), range_facet_l=PostingsFormat(name=NestedPulsing), text=MockFixedIntBlock(blockSize=1731), _version_=Pulsing41(freqCutoff=6 minBlockSize=38 maxBlockSize=88), val_i=MockVariableIntBlock(baseBlockSize=104), SubjectTerms_mfacet=Pulsing41(freqCutoff=6 minBlockSize=38 maxBlockSize=88), series_t=Pulsing41(freqCutoff=6 minBlockSize=38 maxBlockSize=88), a_t=PostingsFormat(name=NestedPulsing), n_tdt1=Pulsing41(freqCutoff=6 minBlockSize=38 maxBlockSize=88), regex_dup_A_s=MockFixedIntBlock(blockSize=1731), price=MockFixedIntBlock(blockSize=1731), other_tl1=Pulsing41(freqCutoff=6 minBlockSize=38 maxBlockSize=88), n_l1=PostingsFormat(name=NestedPulsing), a_si=Pulsing41(freqCutoff=6 minBlockSize=38 maxBlockSize=88), inStock=MockFixedIntBlock(blockSize=1731)}, sim=DefaultSimilarity, locale=sr_ME_#Latn, timezone=BST
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=1,free=300486144,total=523239424
[junit4:junit4]   2> NOTE: All tests run in this JVM: [ResponseLogComponentTest, TestXIncludeConfig, TestDefaultSimilarityFactory, FileBasedSpellCheckerTest, SolrCoreCheckLockOnStartupTest, TestFastOutputStream, DistanceFunctionTest, CacheHeaderTest, TestFiltering, MBeansHandlerTest, TermVectorComponentDistributedTest, SpellCheckCollatorTest, DistributedTermsComponentTest, TestRandomFaceting, TestQuerySenderNoQuery, TestConfig, RequestHandlersTest, TestWordDelimiterFilterFactory, DefaultValueUpdateProcessorTest, ChaosMonkeySafeLeaderTest, TestStressVersions, ChaosMonkeyNothingIsSafeTest, SyncSliceTest, TestCoreContainer, SolrPluginUtilsTest, TestAnalyzedSuggestions, TestIndexingPerformance, TestAtomicUpdateErrorCases, TestLuceneMatchVersion, FieldAnalysisRequestHandlerTest, TestExtendedDismaxParser, CSVRequestHandlerTest, CopyFieldTest, TestLFUCache, CoreAdminHandlerTest, SystemInfoHandlerTest, TestQueryTypes, NoCacheHeaderTest, TestJoin, SpatialFilterTest, SuggesterWFSTTest, BasicDistributedZkTest]
[junit4:junit4] Completed on J0 in 532.72s, 1 test, 1 failure <<< FAILURES!

[...truncated 80 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:346: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/common-build.xml:428: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:1176: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:841: There were test failures: 248 suites, 1037 tests, 1 failure, 23 ignored (5 assumptions)

Total time: 41 minutes 20 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-trunk-java7 - Build # 3696 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-java7/3696/

2 tests failed.
FAILED:  org.apache.solr.cloud.BasicDistributedZkTest.testDistribSearch

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([6831F93F68701A73]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.BasicDistributedZkTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([6831F93F68701A73]:0)




Build Log:
[...truncated 9515 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
[junit4:junit4]   2> 0 T1186 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4]   2> 6 T1186 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZkTest-1359654316331
[junit4:junit4]   2> 7 T1186 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 8 T1187 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 11 T1187 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 11 T1187 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4]   2> 11 T1187 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4]   2> 12 T1187 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 13 T1187 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4]   2> 109 T1186 oasc.ZkTestServer.run start zk server on port:33325
[junit4:junit4]   2> 109 T1186 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:33325 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@793ff194
[junit4:junit4]   2> 110 T1186 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 111 T1192 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:33325. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 111 T1192 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:33325, initiating session
[junit4:junit4]   2> 112 T1188 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:23577
[junit4:junit4]   2> 112 T1188 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:23577
[junit4:junit4]   2> 113 T1190 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 149 T1190 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c91b669320000 with negotiated timeout 10000 for client /140.211.11.196:23577
[junit4:junit4]   2> 149 T1192 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:33325, sessionid = 0x13c91b669320000, negotiated timeout = 10000
[junit4:junit4]   2> 149 T1193 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@793ff194 name:ZooKeeperConnection Watcher:127.0.0.1:33325 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 149 T1186 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 150 T1186 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 154 T1191 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c91b669320000
[junit4:junit4]   2> 170 T1193 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 170 T1188 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:23577 which had sessionid 0x13c91b669320000
[junit4:junit4]   2> 170 T1186 oaz.ZooKeeper.close Session: 0x13c91b669320000 closed
[junit4:junit4]   2> 171 T1186 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:33325/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@583849b8
[junit4:junit4]   2> 172 T1186 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 172 T1194 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:33325. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 173 T1194 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:33325, initiating session
[junit4:junit4]   2> 173 T1188 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:36151
[junit4:junit4]   2> 173 T1188 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:36151
[junit4:junit4]   2> 204 T1190 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c91b669320001 with negotiated timeout 10000 for client /140.211.11.196:36151
[junit4:junit4]   2> 204 T1194 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:33325, sessionid = 0x13c91b669320001, negotiated timeout = 10000
[junit4:junit4]   2> 204 T1195 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@583849b8 name:ZooKeeperConnection Watcher:127.0.0.1:33325/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 205 T1186 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 205 T1186 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 221 T1186 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 226 T1186 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 260 T1186 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 265 T1186 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 266 T1186 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 279 T1186 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 280 T1186 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 404 T1186 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 405 T1186 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 410 T1186 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 411 T1186 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 415 T1186 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 416 T1186 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 420 T1186 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 421 T1186 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 425 T1186 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 426 T1186 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 430 T1186 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 431 T1186 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 435 T1186 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 436 T1186 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 440 T1191 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c91b669320001
[junit4:junit4]   2> 441 T1195 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 441 T1186 oaz.ZooKeeper.close Session: 0x13c91b669320001 closed
[junit4:junit4]   2> 442 T1188 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:36151 which had sessionid 0x13c91b669320001
[junit4:junit4]   2> 611 T1186 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 615 T1186 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:27671
[junit4:junit4]   2> 616 T1186 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 617 T1186 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 617 T1186 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359654316766
[junit4:junit4]   2> 618 T1186 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359654316766/solr.xml
[junit4:junit4]   2> 618 T1186 oasc.CoreContainer.<init> New CoreContainer 1717344474
[junit4:junit4]   2> 619 T1186 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359654316766/'
[junit4:junit4]   2> 620 T1186 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359654316766/'
[junit4:junit4]   2> 675 T1186 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 676 T1186 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 676 T1186 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 677 T1186 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 678 T1186 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 678 T1186 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 679 T1186 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 679 T1186 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 680 T1186 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 680 T1186 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 703 T1186 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 723 T1186 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:33325/solr
[junit4:junit4]   2> 724 T1186 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 724 T1186 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:33325 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@46573acf
[junit4:junit4]   2> 725 T1186 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 726 T1205 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:33325. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 726 T1205 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:33325, initiating session
[junit4:junit4]   2> 727 T1188 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:17081
[junit4:junit4]   2> 727 T1188 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:17081
[junit4:junit4]   2> 733 T1190 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c91b669320002 with negotiated timeout 20000 for client /140.211.11.196:17081
[junit4:junit4]   2> 733 T1205 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:33325, sessionid = 0x13c91b669320002, negotiated timeout = 20000
[junit4:junit4]   2> 733 T1206 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@46573acf name:ZooKeeperConnection Watcher:127.0.0.1:33325 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 734 T1186 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 735 T1191 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c91b669320002
[junit4:junit4]   2> 736 T1206 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 736 T1186 oaz.ZooKeeper.close Session: 0x13c91b669320002 closed
[junit4:junit4]   2> 736 T1188 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:17081 which had sessionid 0x13c91b669320002
[junit4:junit4]   2> 737 T1186 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 750 T1186 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:33325/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@2c832710
[junit4:junit4]   2> 751 T1186 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 751 T1207 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:33325. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 752 T1207 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:33325, initiating session
[junit4:junit4]   2> 752 T1188 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:11257
[junit4:junit4]   2> 753 T1188 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:11257
[junit4:junit4]   2> 754 T1190 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c91b669320003 with negotiated timeout 20000 for client /140.211.11.196:11257
[junit4:junit4]   2> 754 T1207 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:33325, sessionid = 0x13c91b669320003, negotiated timeout = 20000
[junit4:junit4]   2> 755 T1208 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2c832710 name:ZooKeeperConnection Watcher:127.0.0.1:33325/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 755 T1186 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 757 T1191 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c91b669320003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 766 T1186 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 769 T1186 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:27671_
[junit4:junit4]   2> 770 T1191 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c91b669320003 type:delete cxid:0x7 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:27671_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:27671_
[junit4:junit4]   2> 771 T1186 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:27671_
[junit4:junit4]   2> 780 T1186 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 794 T1191 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c91b669320003 type:delete cxid:0x15 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 796 T1186 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 799 T1186 oasc.Overseer.start Overseer (id=89106305275527171-127.0.0.1:27671_-n_0000000000) starting
[junit4:junit4]   2> 800 T1191 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c91b669320003 type:create cxid:0x1a zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 807 T1191 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c91b669320003 type:create cxid:0x1b zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 808 T1191 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c91b669320003 type:create cxid:0x1c zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 815 T1191 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c91b669320003 type:create cxid:0x1d zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 816 T1210 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 818 T1186 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 822 T1186 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 829 T1211 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359654316766/collection1
[junit4:junit4]   2> 830 T1209 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 830 T1211 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 831 T1211 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 831 T1211 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 833 T1211 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359654316766/collection1/'
[junit4:junit4]   2> 834 T1211 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359654316766/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 835 T1211 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359654316766/collection1/lib/README' to classloader
[junit4:junit4]   2> 888 T1211 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 960 T1211 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 1061 T1211 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 1067 T1211 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 1702 T1211 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1710 T1211 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1714 T1211 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1729 T1211 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1734 T1211 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1738 T1211 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1740 T1211 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1741 T1211 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1359654316766/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/control/data/
[junit4:junit4]   2> 1741 T1211 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4096d1d9
[junit4:junit4]   2> 1742 T1211 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 1770 T1211 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/control/data forceNew:false
[junit4:junit4]   2> 1771 T1211 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/control/data
[junit4:junit4]   2> 1771 T1211 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/control/data/index/
[junit4:junit4]   2> 1772 T1211 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1773 T1211 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/control/data/index forceNew:false
[junit4:junit4]   2> 1804 T1211 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@36a95c7c),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 1804 T1211 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 1805 T1211 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/control/data/index
[junit4:junit4]   2> 1805 T1211 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 1807 T1211 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1807 T1211 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 1807 T1211 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1808 T1211 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 1808 T1211 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1808 T1211 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1809 T1211 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1809 T1211 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1810 T1211 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1810 T1211 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 1810 T1211 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 1811 T1211 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1811 T1211 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 1811 T1211 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 1812 T1211 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 1812 T1211 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1812 T1211 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1813 T1211 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1813 T1211 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1814 T1211 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1814 T1211 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1814 T1211 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1815 T1211 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1815 T1211 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 1816 T1211 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 1816 T1211 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 1826 T1211 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1830 T1211 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 1832 T1211 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 1834 T1211 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/control/data
[junit4:junit4]   2> 1836 T1211 oass.SolrIndexSearcher.<init> Opening Searcher@34d709b3 main
[junit4:junit4]   2> 1837 T1211 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1837 T1211 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1837 T1211 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 1854 T1211 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 1908 T1212 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@34d709b3 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1911 T1211 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 1911 T1211 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 1912 T1191 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c91b669320003 type:create cxid:0x50 zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 2335 T1209 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2336 T1209 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:27671_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:27671"}
[junit4:junit4]   2> 2336 T1209 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4]   2> 2342 T1209 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 2343 T1191 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c91b669320003 type:create cxid:0x58 zxid:0x2c txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 2364 T1208 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 2925 T1211 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 2925 T1211 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:27671 collection:control_collection shard:shard1
[junit4:junit4]   2> 2926 T1211 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4]   2> 2969 T1191 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c91b669320003 type:delete cxid:0x72 zxid:0x36 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4]   2> 2970 T1211 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 2971 T1191 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c91b669320003 type:create cxid:0x73 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2979 T1211 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 2979 T1211 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 2980 T1211 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:27671/collection1/
[junit4:junit4]   2> 2980 T1211 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 2980 T1211 oasc.SyncStrategy.syncToMe http://127.0.0.1:27671/collection1/ has no replicas
[junit4:junit4]   2> 2980 T1211 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:27671/collection1/
[junit4:junit4]   2> 2981 T1211 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4]   2> 2985 T1191 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c91b669320003 type:create cxid:0x7d zxid:0x3b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3869 T1209 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3925 T1208 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 3965 T1211 oasc.ZkController.register We are http://127.0.0.1:27671/collection1/ and leader is http://127.0.0.1:27671/collection1/
[junit4:junit4]   2> 3965 T1211 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:27671
[junit4:junit4]   2> 3965 T1211 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 3965 T1211 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 3966 T1211 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 3972 T1211 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3973 T1186 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 3974 T1186 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 3974 T1186 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 3986 T1186 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 3998 T1186 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:33325/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@6a67a9bc
[junit4:junit4]   2> 3999 T1186 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 3999 T1213 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:33325. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4000 T1213 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:33325, initiating session
[junit4:junit4]   2> 4000 T1188 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:12207
[junit4:junit4]   2> 4001 T1188 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:12207
[junit4:junit4]   2> 4002 T1190 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c91b669320004 with negotiated timeout 10000 for client /140.211.11.196:12207
[junit4:junit4]   2> 4002 T1213 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:33325, sessionid = 0x13c91b669320004, negotiated timeout = 10000
[junit4:junit4]   2> 4002 T1214 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6a67a9bc name:ZooKeeperConnection Watcher:127.0.0.1:33325/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4003 T1186 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4004 T1186 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 4008 T1186 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 4132 T1186 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 4135 T1186 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:52746
[junit4:junit4]   2> 4135 T1186 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 4135 T1186 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 4136 T1186 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359654320332
[junit4:junit4]   2> 4136 T1186 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359654320332/solr.xml
[junit4:junit4]   2> 4137 T1186 oasc.CoreContainer.<init> New CoreContainer 89929548
[junit4:junit4]   2> 4137 T1186 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359654320332/'
[junit4:junit4]   2> 4137 T1186 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359654320332/'
[junit4:junit4]   2> 4178 T1186 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 4179 T1186 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 4179 T1186 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 4179 T1186 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 4180 T1186 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 4180 T1186 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 4180 T1186 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 4181 T1186 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 4181 T1186 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 4182 T1186 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 4198 T1186 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 4212 T1186 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:33325/solr
[junit4:junit4]   2> 4213 T1186 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 4214 T1186 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:33325 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@34048852
[junit4:junit4]   2> 4214 T1186 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4215 T1224 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:33325. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4215 T1224 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:33325, initiating session
[junit4:junit4]   2> 4216 T1188 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:27517
[junit4:junit4]   2> 4216 T1188 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:27517
[junit4:junit4]   2> 4218 T1190 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c91b669320005 with negotiated timeout 20000 for client /140.211.11.196:27517
[junit4:junit4]   2> 4218 T1224 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:33325, sessionid = 0x13c91b669320005, negotiated timeout = 20000
[junit4:junit4]   2> 4219 T1225 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@34048852 name:ZooKeeperConnection Watcher:127.0.0.1:33325 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4219 T1186 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4220 T1191 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c91b669320005
[junit4:junit4]   2> 4228 T1225 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 4228 T1188 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:27517 which had sessionid 0x13c91b669320005
[junit4:junit4]   2> 4228 T1186 oaz.ZooKeeper.close Session: 0x13c91b669320005 closed
[junit4:junit4]   2> 4229 T1186 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 4242 T1186 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:33325/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@5aa526f4
[junit4:junit4]   2> 4243 T1186 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4243 T1226 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:33325. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4244 T1226 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:33325, initiating session
[junit4:junit4]   2> 4245 T1188 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:44871
[junit4:junit4]   2> 4245 T1188 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:44871
[junit4:junit4]   2> 4247 T1190 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c91b669320006 with negotiated timeout 20000 for client /140.211.11.196:44871
[junit4:junit4]   2> 4247 T1226 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:33325, sessionid = 0x13c91b669320006, negotiated timeout = 20000
[junit4:junit4]   2> 4248 T1227 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5aa526f4 name:ZooKeeperConnection Watcher:127.0.0.1:33325/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4248 T1186 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4249 T1191 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c91b669320006 type:create cxid:0x1 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4250 T1191 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c91b669320006 type:create cxid:0x2 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4258 T1186 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52746_
[junit4:junit4]   2> 4259 T1191 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c91b669320006 type:delete cxid:0x4 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:52746_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:52746_
[junit4:junit4]   2> 4260 T1186 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52746_
[junit4:junit4]   2> 4264 T1208 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 4264 T1214 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 4265 T1208 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 4269 T1186 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 4278 T1228 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359654320332/collection1
[junit4:junit4]   2> 4278 T1228 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 4279 T1228 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 4279 T1228 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 4281 T1228 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359654320332/collection1/'
[junit4:junit4]   2> 4283 T1228 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359654320332/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 4283 T1228 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359654320332/collection1/lib/README' to classloader
[junit4:junit4]   2> 4345 T1228 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 4426 T1228 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 4528 T1228 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 4535 T1228 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 5217 T1228 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 5229 T1228 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 5234 T1228 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 5249 T1228 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5255 T1228 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 5265 T1228 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5267 T1228 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 5268 T1228 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1359654320332/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty1/
[junit4:junit4]   2> 5269 T1228 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4096d1d9
[junit4:junit4]   2> 5269 T1228 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 5270 T1228 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty1 forceNew:false
[junit4:junit4]   2> 5271 T1228 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty1
[junit4:junit4]   2> 5271 T1228 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty1/index/
[junit4:junit4]   2> 5272 T1228 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 5273 T1228 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty1/index forceNew:false
[junit4:junit4]   2> 5327 T1228 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4c63363e),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 5327 T1228 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 5328 T1228 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty1/index
[junit4:junit4]   2> 5329 T1228 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 5331 T1228 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 5331 T1228 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 5332 T1228 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 5332 T1228 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 5333 T1228 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 5333 T1228 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 5334 T1228 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 5335 T1228 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 5335 T1228 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 5336 T1228 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 5337 T1228 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 5337 T1228 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 5338 T1228 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 5338 T1228 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 5339 T1228 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 5339 T1228 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 5340 T1228 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5341 T1228 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5341 T1228 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5342 T1228 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5343 T1228 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5343 T1228 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5344 T1228 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5345 T1228 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 5345 T1228 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 5346 T1228 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 5360 T1228 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 5365 T1228 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 5366 T1228 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 5369 T1228 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty1
[junit4:junit4]   2> 5371 T1228 oass.SolrIndexSearcher.<init> Opening Searcher@5a894e80 main
[junit4:junit4]   2> 5372 T1228 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 5373 T1228 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 5373 T1228 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 5381 T1228 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 5440 T1209 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 5441 T1209 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:27671_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:27671"}
[junit4:junit4]   2> 5442 T1229 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5a894e80 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 5448 T1228 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 5449 T1228 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 5454 T1227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5454 T1214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5454 T1208 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6966 T1209 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6968 T1209 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:52746_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:52746"}
[junit4:junit4]   2> 6968 T1209 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4]   2> 6969 T1209 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 6976 T1208 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6976 T1214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6976 T1227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7457 T1228 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 7457 T1228 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:52746 collection:collection1 shard:shard1
[junit4:junit4]   2> 7458 T1228 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 7469 T1191 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c91b669320006 type:delete cxid:0x45 zxid:0x5b txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 7478 T1228 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 7478 T1191 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c91b669320006 type:create cxid:0x46 zxid:0x5c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7487 T1228 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 7487 T1228 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 7487 T1228 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:52746/collection1/
[junit4:junit4]   2> 7488 T1228 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 7488 T1228 oasc.SyncStrategy.syncToMe http://127.0.0.1:52746/collection1/ has no replicas
[junit4:junit4]   2> 7488 T1228 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:52746/collection1/
[junit4:junit4]   2> 7489 T1228 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 7495 T1191 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c91b669320006 type:create cxid:0x50 zxid:0x60 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8488 T1209 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8532 T1214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8532 T1208 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8532 T1227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8579 T1228 oasc.ZkController.register We are http://127.0.0.1:52746/collection1/ and leader is http://127.0.0.1:52746/collection1/
[junit4:junit4]   2> 8579 T1228 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:52746
[junit4:junit4]   2> 8580 T1228 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 8580 T1228 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 8580 T1228 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 8584 T1228 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8585 T1186 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 8586 T1186 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 8587 T1186 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 8731 T1186 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 8734 T1186 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:58857
[junit4:junit4]   2> 8735 T1186 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 8735 T1186 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 8735 T1186 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359654324927
[junit4:junit4]   2> 8736 T1186 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359654324927/solr.xml
[junit4:junit4]   2> 8736 T1186 oasc.CoreContainer.<init> New CoreContainer 444852282
[junit4:junit4]   2> 8737 T1186 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359654324927/'
[junit4:junit4]   2> 8737 T1186 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359654324927/'
[junit4:junit4]   2> 8779 T1186 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 8780 T1186 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 8780 T1186 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 8780 T1186 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 8781 T1186 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 8781 T1186 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 8781 T1186 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 8782 T1186 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 8782 T1186 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 8783 T1186 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 8799 T1186 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 8814 T1186 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:33325/solr
[junit4:junit4]   2> 8815 T1186 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 8815 T1186 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:33325 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@18aea76f
[junit4:junit4]   2> 8817 T1186 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 8817 T1239 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:33325. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 8818 T1239 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:33325, initiating session
[junit4:junit4]   2> 8818 T1188 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:41990
[junit4:junit4]   2> 8819 T1188 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:41990
[junit4:junit4]   2> 8827 T1190 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c91b669320007 with negotiated timeout 20000 for client /140.211.11.196:41990
[junit4:junit4]   2> 8827 T1239 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:33325, sessionid = 0x13c91b669320007, negotiated timeout = 20000
[junit4:junit4]   2> 8828 T1240 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@18aea76f name:ZooKeeperConnection Watcher:127.0.0.1:33325 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 8828 T1186 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 8830 T1191 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c91b669320007
[junit4:junit4]   2> 8836 T1240 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 8836 T1188 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:41990 which had sessionid 0x13c91b669320007
[junit4:junit4]   2> 8835 T1186 oaz.ZooKeeper.close Session: 0x13c91b669320007 closed
[junit4:junit4]   2> 8837 T1186 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 8846 T1186 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:33325/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@1a5418a7
[junit4:junit4]   2> 8847 T1186 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 8847 T1241 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:33325. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 8848 T1241 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:33325, initiating session
[junit4:junit4]   2> 8849 T1188 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:62464
[junit4:junit4]   2> 8849 T1188 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:62464
[junit4:junit4]   2> 8851 T1190 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c91b669320008 with negotiated timeout 20000 for client /140.211.11.196:62464
[junit4:junit4]   2> 8851 T1241 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:33325, sessionid = 0x13c91b669320008, negotiated timeout = 20000
[junit4:junit4]   2> 8852 T1242 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a5418a7 name:ZooKeeperConnection Watcher:127.0.0.1:33325/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 8852 T1186 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 8853 T1191 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c91b669320008 type:create cxid:0x1 zxid:0x6d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8855 T1191 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c91b669320008 type:create cxid:0x2 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8862 T1186 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58857_
[junit4:junit4]   2> 8863 T1191 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c91b669320008 type:delete cxid:0x4 zxid:0x6f txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:58857_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:58857_
[junit4:junit4]   2> 8865 T1186 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:58857_
[junit4:junit4]   2> 8868 T1214 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 8868 T1227 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 8869 T1208 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 8869 T1227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8869 T1214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8869 T1208 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 8875 T1186 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 8882 T1243 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359654324927/collection1
[junit4:junit4]   2> 8882 T1243 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 8883 T1243 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 8884 T1243 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 8886 T1243 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359654324927/collection1/'
[junit4:junit4]   2> 8887 T1243 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359654324927/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 8888 T1243 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359654324927/collection1/lib/README' to classloader
[junit4:junit4]   2> 8948 T1243 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 9027 T1243 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 9128 T1243 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 9134 T1243 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 9817 T1243 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 9829 T1243 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 9834 T1243 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 9859 T1243 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 9865 T1243 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 9871 T1243 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 9873 T1243 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 9874 T1243 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1359654324927/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty2/
[junit4:junit4]   2> 9875 T1243 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4096d1d9
[junit4:junit4]   2> 9876 T1243 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 9877 T1243 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty2 forceNew:false
[junit4:junit4]   2> 9877 T1243 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty2
[junit4:junit4]   2> 9878 T1243 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty2/index/
[junit4:junit4]   2> 9879 T1243 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 9879 T1243 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty2/index forceNew:false
[junit4:junit4]   2> 9905 T1243 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5dd21f15),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 9906 T1243 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 9907 T1243 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty2/index
[junit4:junit4]   2> 9907 T1243 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 9910 T1243 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 9910 T1243 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 9911 T1243 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 9911 T1243 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 9912 T1243 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 9912 T1243 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 9913 T1243 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 9914 T1243 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 9914 T1243 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 9915 T1243 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 9916 T1243 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 9916 T1243 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 9917 T1243 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 9917 T1243 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 9918 T1243 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 9918 T1243 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 9919 T1243 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9920 T1243 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9920 T1243 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9921 T1243 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9922 T1243 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9923 T1243 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9923 T1243 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9924 T1243 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 9925 T1243 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 9925 T1243 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 9939 T1243 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 9944 T1243 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 9946 T1243 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 9949 T1243 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty2
[junit4:junit4]   2> 9950 T1243 oass.SolrIndexSearcher.<init> Opening Searcher@12bec43e main
[junit4:junit4]   2> 9952 T1243 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 9952 T1243 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 9953 T1243 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 9984 T1243 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 10041 T1209 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 10042 T1209 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:52746_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:52746"}
[junit4:junit4]   2> 10129 T1208 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10129 T1242 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10129 T1214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10129 T1227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10165 T1244 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@12bec43e main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 10171 T1243 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 10171 T1243 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 11644 T1209 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11646 T1209 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:58857_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:58857"}
[junit4:junit4]   2> 11646 T1209 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 11646 T1209 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 11664 T1227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11665 T1208 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11664 T1214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11665 T1242 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 12175 T1243 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 12175 T1243 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:58857 collection:collection1 shard:shard2
[junit4:junit4]   2> 12177 T1243 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4]   2> 12193 T1191 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c91b669320008 type:delete cxid:0x44 zxid:0x7e txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders/shard2 Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders/shard2
[junit4:junit4]   2> 12195 T1243 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 12196 T1191 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c91b669320008 type:create cxid:0x45 zxid:0x7f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12206 T1243 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 12207 T1243 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 12207 T1243 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:58857/collection1/
[junit4:junit4]   2> 12207 T1243 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 12208 T1243 oasc.SyncStrategy.syncToMe http://127.0.0.1:58857/collection1/ has no replicas
[junit4:junit4]   2> 12208 T1243 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:58857/collection1/
[junit4:junit4]   2> 12208 T1243 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4]   2> 12213 T1191 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c91b669320008 type:create cxid:0x4e zxid:0x82 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 13171 T1209 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 13193 T1242 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13193 T1214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13193 T1208 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13193 T1227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13243 T1243 oasc.ZkController.register We are http://127.0.0.1:58857/collection1/ and leader is http://127.0.0.1:58857/collection1/
[junit4:junit4]   2> 13243 T1243 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:58857
[junit4:junit4]   2> 13243 T1243 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 13244 T1243 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 13244 T1243 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 13246 T1243 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 13248 T1186 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 13248 T1186 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 13249 T1186 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 13380 T1186 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 13383 T1186 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:56708
[junit4:junit4]   2> 13383 T1186 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 13384 T1186 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 13384 T1186 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359654329583
[junit4:junit4]   2> 13385 T1186 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359654329583/solr.xml
[junit4:junit4]   2> 13385 T1186 oasc.CoreContainer.<init> New CoreContainer 408572808
[junit4:junit4]   2> 13385 T1186 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359654329583/'
[junit4:junit4]   2> 13386 T1186 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359654329583/'
[junit4:junit4]   2> 13427 T1186 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 13427 T1186 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 13428 T1186 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 13428 T1186 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 13428 T1186 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 13429 T1186 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 13429 T1186 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 13430 T1186 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 13430 T1186 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 13430 T1186 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 13445 T1186 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 13460 T1186 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:33325/solr
[junit4:junit4]   2> 13460 T1186 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 13461 T1186 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:33325 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@4745123d
[junit4:junit4]   2> 13462 T1186 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 13462 T1254 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:33325. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 13463 T1254 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:33325, initiating session
[junit4:junit4]   2> 13463 T1188 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:25715
[junit4:junit4]   2> 13463 T1188 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:25715
[junit4:junit4]   2> 13466 T1190 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c91b669320009 with negotiated timeout 20000 for client /140.211.11.196:25715
[junit4:junit4]   2> 13466 T1254 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:33325, sessionid = 0x13c91b669320009, negotiated timeout = 20000
[junit4:junit4]   2> 13467 T1255 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4745123d name:ZooKeeperConnection Watcher:127.0.0.1:33325 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 13467 T1186 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 13469 T1191 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c91b669320009
[junit4:junit4]   2> 13476 T1255 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 13476 T1186 oaz.ZooKeeper.close Session: 0x13c91b669320009 closed
[junit4:junit4]   2> 13476 T1188 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:25715 which had sessionid 0x13c91b669320009
[junit4:junit4]   2> 13477 T1186 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 13484 T1186 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:33325/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@50c0fa32
[junit4:junit4]   2> 13485 T1186 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 13485 T1256 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:33325. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 13486 T1256 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:33325, initiating session
[junit4:junit4]   2> 13486 T1188 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:20623
[junit4:junit4]   2> 13486 T1188 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:20623
[junit4:junit4]   2> 13487 T1190 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c91b66932000a with negotiated timeout 20000 for client /140.211.11.196:20623
[junit4:junit4]   2> 13487 T1256 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:33325, sessionid = 0x13c91b66932000a, negotiated timeout = 20000
[junit4:junit4]   2> 13488 T1257 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@50c0fa32 name:ZooKeeperConnection Watcher:127.0.0.1:33325/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 13488 T1186 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 13489 T1191 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c91b66932000a type:create cxid:0x1 zxid:0x8f txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 13490 T1191 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c91b66932000a type:create cxid:0x2 zxid:0x90 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 13498 T1186 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56708_
[junit4:junit4]   2> 13499 T1191 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c91b66932000a type:delete cxid:0x4 zxid:0x91 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:56708_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:56708_
[junit4:junit4]   2> 13501 T1186 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:56708_
[junit4:junit4]   2> 13504 T1214 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 13504 T1208 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 13505 T1242 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 13504 T1214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13505 T1208 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13505 T1227 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 13505 T1242 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13506 T1227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13513 T1186 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 13522 T1258 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359654329583/collection1
[junit4:junit4]   2> 13522 T1258 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 13523 T1258 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 13523 T1258 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 13525 T1258 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359654329583/collection1/'
[junit4:junit4]   2> 13526 T1258 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359654329583/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 13527 T1258 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359654329583/collection1/lib/README' to classloader
[junit4:junit4]   2> 13585 T1258 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 13664 T1258 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 13765 T1258 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 13772 T1258 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 14455 T1258 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 14467 T1258 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 14471 T1258 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 14487 T1258 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 14492 T1258 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 14497 T1258 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 14498 T1258 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 14500 T1258 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1359654329583/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty3/
[junit4:junit4]   2> 14500 T1258 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4096d1d9
[junit4:junit4]   2> 14501 T1258 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 14502 T1258 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty3 forceNew:false
[junit4:junit4]   2> 14503 T1258 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty3
[junit4:junit4]   2> 14503 T1258 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty3/index/
[junit4:junit4]   2> 14504 T1258 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 14504 T1258 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty3/index forceNew:false
[junit4:junit4]   2> 14510 T1258 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4b93ccbb),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 14510 T1258 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 14511 T1258 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty3/index
[junit4:junit4]   2> 14512 T1258 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 14514 T1258 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 14515 T1258 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 14515 T1258 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 14516 T1258 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 14516 T1258 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 14517 T1258 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 14517 T1258 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 14518 T1258 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 14519 T1258 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 14519 T1258 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 14520 T1258 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 14521 T1258 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 14521 T1258 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 14521 T1258 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 14522 T1258 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 14523 T1258 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 14523 T1258 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14524 T1258 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14525 T1258 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14525 T1258 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14526 T1258 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14527 T1258 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14527 T1258 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14528 T1258 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 14529 T1258 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 14529 T1258 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 14549 T1258 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 14555 T1258 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 14556 T1258 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 14559 T1258 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty3
[junit4:junit4]   2> 14561 T1258 oass.SolrIndexSearcher.<init> Opening Searcher@2ccb1ea4 main
[junit4:junit4]   2> 14562 T1258 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 14562 T1258 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 14563 T1258 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 14571 T1258 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 14632 T1259 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2ccb1ea4 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 14638 T1258 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 14638 T1258 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 14707 T1209 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14708 T1209 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:58857_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:58857"}
[junit4:junit4]   2> 14721 T1209 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:56708_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:56708"}
[junit4:junit4]   2> 14721 T1209 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 14721 T1209 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 14734 T1227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14734 T1242 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14734 T1214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14734 T1257 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14734 T1208 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15642 T1258 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 15642 T1258 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:56708 collection:collection1 shard:shard1
[junit4:junit4]   2> 15671 T1258 oasc.ZkController.register We are http://127.0.0.1:56708/collection1/ and leader is http://127.0.0.1:52746/collection1/
[junit4:junit4]   2> 15672 T1258 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:56708
[junit4:junit4]   2> 15672 T1258 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 15672 T1258 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C116 name=collection1 org.apache.solr.core.SolrCore@701e0b2e url=http://127.0.0.1:56708/collection1 node=127.0.0.1:56708_ C116_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:56708_, base_url=http://127.0.0.1:56708}
[junit4:junit4]   2> 15680 T1260 C116 P56708 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 15680 T1258 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 15681 T1260 C116 P56708 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 15681 T1260 C116 P56708 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 15682 T1260 C116 P56708 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 15682 T1186 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 15683 T1186 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 15683 T1186 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 15683 T1260 C116 P56708 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 15821 T1186 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 15824 T1186 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:50178
[junit4:junit4]   2> 15824 T1186 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 15825 T1186 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 15825 T1186 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359654332022
[junit4:junit4]   2> 15825 T1186 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359654332022/solr.xml
[junit4:junit4]   2> 15826 T1186 oasc.CoreContainer.<init> New CoreContainer 1333076254
[junit4:junit4]   2> 15826 T1186 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359654332022/'
[junit4:junit4]   2> 15827 T1186 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359654332022/'
[junit4:junit4]   2> 15868 T1186 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 60000
[junit4:junit4]   2> 15868 T1186 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 15869 T1186 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 15869 T1186 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 15869 T1186 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 15870 T1186 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 15870 T1186 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 15870 T1186 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 15871 T1186 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 15871 T1186 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=60000&connTimeout=15000&retry=false
[junit4:junit4]   2> 15886 T1186 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 15900 T1186 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:33325/solr
[junit4:junit4]   2> 15900 T1186 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 15901 T1186 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:33325 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@1e5467fb
[junit4:junit4]   2> 15901 T1186 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 15902 T1270 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:33325. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 15903 T1270 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:33325, initiating session
[junit4:junit4]   2> 15903 T1188 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:31649
[junit4:junit4]   2> 15903 T1188 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:31649
[junit4:junit4]   2> 15906 T1190 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c91b66932000b with negotiated timeout 20000 for client /140.211.11.196:31649
[junit4:junit4]   2> 15906 T1270 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:33325, sessionid = 0x13c91b66932000b, negotiated timeout = 20000
[junit4:junit4]   2> 15906 T1271 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1e5467fb name:ZooKeeperConnection Watcher:127.0.0.1:33325 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 15906 T1186 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 15908 T1191 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13c91b66932000b
[junit4:junit4]   2> 15909 T1271 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 15909 T1186 oaz.ZooKeeper.close Session: 0x13c91b66932000b closed
[junit4:junit4]   2> 15909 T1188 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:31649 which had sessionid 0x13c91b66932000b
[junit4:junit4]   2> 15909 T1186 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=60000&connTimeout=15000
[junit4:junit4]   2> 15917 T1186 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:33325/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@57288854
[junit4:junit4]   2> 15918 T1186 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 15918 T1272 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:33325. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 15918 T1272 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:33325, initiating session
[junit4:junit4]   2> 15919 T1188 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:29130
[junit4:junit4]   2> 15919 T1188 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:29130
[junit4:junit4]   2> 15921 T1190 oazs.ZooKeeperServer.finishSessionInit Established session 0x13c91b66932000c with negotiated timeout 20000 for client /140.211.11.196:29130
[junit4:junit4]   2> 15921 T1272 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:33325, sessionid = 0x13c91b66932000c, negotiated timeout = 20000
[junit4:junit4]   2> 15921 T1273 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@57288854 name:ZooKeeperConnection Watcher:127.0.0.1:33325/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 15921 T1186 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 15922 T1191 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c91b66932000c type:create cxid:0x1 zxid:0xa1 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 15924 T1191 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c91b66932000c type:create cxid:0x2 zxid:0xa2 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 15925 T1186 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50178_
[junit4:junit4]   2> 15926 T1191 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13c91b66932000c type:delete cxid:0x4 zxid:0xa3 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:50178_ Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:50178_
[junit4:junit4]   2> 15934 T1186 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50178_
[junit4:junit4]   2> 15937 T1214 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 15937 T1208 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 15938 T1257 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 15938 T1208 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15938 T1242 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 15938 T1227 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 15937 T1214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15939 T1227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15938 T1242 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15938 T1257 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 15942 T1186 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 15950 T1274 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359654332022/collection1
[junit4:junit4]   2> 15951 T1274 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 15951 T1274 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 15951 T1274 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 15953 T1274 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359654332022/collection1/'
[junit4:junit4]   2> 15953 T1274 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359654332022/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 15954 T1274 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359654332022/collection1/lib/README' to classloader
[junit4:junit4]   2> 16005 T1274 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 16055 T1274 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 16156 T1274 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 16160 T1274 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 16242 T1209 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16244 T1209 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:56708_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:56708"}
[junit4:junit4]   2> 16258 T1208 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16258 T1227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16258 T1242 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16258 T1214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16258 T1257 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16258 T1273 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16604 T1274 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 16612 T1274 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 16615 T1274 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 16629 T1274 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 16633 T1274 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 16637 T1274 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 16638 T1274 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 16639 T1274 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-jetty4-1359654332022/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty4/
[junit4:junit4]   2> 16640 T1274 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4096d1d9
[junit4:junit4]   2> 16640 T1274 oasc.SolrCore.initDirectoryFactory solr.StandardDirectoryFactory
[junit4:junit4]   2> 16641 T1274 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty4 forceNew:false
[junit4:junit4]   2> 16641 T1274 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty4
[junit4:junit4]   2> 16641 T1274 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty4/index/
[junit4:junit4]   2> 16642 T1274 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 16642 T1274 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty4/index forceNew:false
[junit4:junit4]   2> 16656 T1274 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2e16c743),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 16657 T1274 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 16657 T1274 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty4/index
[junit4:junit4]   2> 16657 T1274 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
[junit4:junit4]   2> 16659 T1274 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 16659 T1274 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
[junit4:junit4]   2> 16660 T1274 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 16660 T1274 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
[junit4:junit4]   2> 16661 T1274 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 16661 T1274 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 16661 T1274 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 16662 T1274 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 16662 T1274 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 16663 T1274 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
[junit4:junit4]   2> 16663 T1274 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
[junit4:junit4]   2> 16663 T1274 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 16664 T1274 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
[junit4:junit4]   2> 16664 T1274 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
[junit4:junit4]   2> 16664 T1274 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
[junit4:junit4]   2> 16665 T1274 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 16665 T1274 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16666 T1274 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16666 T1274 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16667 T1274 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16667 T1274 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16668 T1274 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16668 T1274 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16668 T1274 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
[junit4:junit4]   2> 16669 T1274 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
[junit4:junit4]   2> 16669 T1274 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
[junit4:junit4]   2> 16678 T1274 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 16681 T1274 oasc.SolrCore.initDeprecatedSupport WARNING solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
[junit4:junit4]   2> 16682 T1274 oasc.SolrCore.initDeprecatedSupport WARNING adding ShowFileRequestHandler with hidden files: [SCHEMA.XML, OLD_SYNONYMS.TXT, STOPWORDS.TXT, PROTWORDS.TXT, OPEN-EXCHANGE-RATES.JSON, SYNONYMS.TXT, CURRENCY.XML, MAPPING-ISOLATIN1ACCENT.TXT]
[junit4:junit4]   2> 16684 T1274 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty4
[junit4:junit4]   2> 16685 T1274 oass.SolrIndexSearcher.<init> Opening Searcher@79ba1ec8 main
[junit4:junit4]   2> 16686 T1274 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 16687 T1274 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 16687 T1274 oashc.SpellCheckComponent.inform Initializing spell checkers
[junit4:junit4]   2> 16692 T1274 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4:junit4]   2> 16740 T1275 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@79ba1ec8 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 16744 T1274 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 16745 T1274 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 17765 T1209 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 17766 T1209 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:50178_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:50178"}
[junit4:junit4]   2> 17767 T1209 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4]   2> 17767 T1209 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
[junit4:junit4]   2> 17797 T1227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17797 T1214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17797 T1257 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17797 T1242 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17797 T1208 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 17797 T1273 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2>  C116_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:56708_, base_url=http://127.0.0.1:56708}
[junit4:junit4]   2> 18713 T1260 C116 P56708 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:52746/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 18713 T1260 C116 P56708 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:56708 START replicas=[http://127.0.0.1:52746/collection1/] nUpdates=100
[junit4:junit4]   2> 18714 T1260 C116 P56708 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 18714 T1260 C116 P56708 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 18715 T1260 C116 P56708 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 18715 T1260 C116 P56708 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 18715 T1260 C116 P56708 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 18716 T1260 C116 P56708 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:52746/collection1/. core=collection1
[junit4:junit4]   2> 18716 T1260 C116 P56708 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C117 name=collection1 org.apache.solr.core.SolrCore@4505d459 url=http://127.0.0.1:52746/collection1 node=127.0.0.1:52746_ C117_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:52746_, base_url=http://127.0.0.1:52746, leader=true}
[junit4:junit4]   2> 18723 T1218 C117 P52746 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 18736 T1219 C117 P52746 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 18737 T1219 C117 P52746 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty1
[junit4:junit4]   2> 18740 T1219 C117 P52746 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4c63363e),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 18741 T1219 C117 P52746 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 18744 T1219 C117 P52746 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4c63363e),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@4c63363e),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 18744 T1219 C117 P52746 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 18745 T1219 C117 P52746 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty1
[junit4:junit4]   2> 18745 T1219 C117 P52746 oass.SolrIndexSearcher.<init> Opening Searcher@750274c0 realtime
[junit4:junit4]   2> 18745 T1219 C117 P52746 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 18746 T1219 C117 P52746 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 10
[junit4:junit4]   2> 18747 T1260 C116 P56708 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 18747 T1260 C116 P56708 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 18748 T1274 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 18748 T1274 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:50178 collection:collection1 shard:shard2
[junit4:junit4]   2> 18755 T1220 C117 P52746 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 18756 T1220 C117 P52746 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=7 
[junit4:junit4]   2> 18757 T1260 C116 P56708 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 18757 T1274 oasc.ZkController.register We are http://127.0.0.1:50178/collection1/ and leader is http://127.0.0.1:58857/collection1/
[junit4:junit4]   2> 18757 T1260 C116 P56708 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 18758 T1274 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:50178
[junit4:junit4]   2> 18758 T1260 C116 P56708 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 18758 T1274 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 18759 T1274 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C118 name=collection1 org.apache.solr.core.SolrCore@33742ed5 url=http://127.0.0.1:50178/collection1 node=127.0.0.1:50178_ C118_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:50178_, base_url=http://127.0.0.1:50178}
[junit4:junit4]   2> 18759 T1277 C118 P50178 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 18759 T1274 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 18760 T1277 C118 P50178 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 18760 T1277 C118 P50178 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 18760 T1277 C118 P50178 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 18761 T1186 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1
[junit4:junit4]   2> 18761 T1220 C117 P52746 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty1
[junit4:junit4]   2> 18761 T1186 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 18762 T1186 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 18762 T1220 C117 P52746 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty1/index
[junit4:junit4]   2> 18763 T1277 C118 P50178 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 18763 T1220 C117 P52746 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=2 
[junit4:junit4]   2> 18764 T1260 C116 P56708 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 18765 T1260 C116 P56708 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty3/index.20130131174535088 forceNew:false
[junit4:junit4]   2> 18765 T1260 C116 P56708 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty3
[junit4:junit4]   2> 18766 T1260 C116 P56708 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty3/index.20130131174535088 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2d017f57) fullCopy=true
[junit4:junit4]   2> 18769 T1220 C117 P52746 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> 18771 T1260 C116 P56708 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty3/index
[junit4:junit4]   2> 18771 T1260 C116 P56708 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 18771 T1260 C116 P56708 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130131174535088
[junit4:junit4]   2> 18772 T1260 C116 P56708 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty3
[junit4:junit4]   2> 18773 T1260 C116 P56708 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty3
[junit4:junit4]   2> 18773 T1260 C116 P56708 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty3/index/ new=./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty3/index.20130131174535088
[junit4:junit4]   2> 18776 T1260 C116 P56708 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty3/index.20130131174535088 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2d017f57),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 18776 T1260 C116 P56708 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 18777 T1260 C116 P56708 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty3
[junit4:junit4]   2> 18777 T1260 C116 P56708 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 18777 T1260 C116 P56708 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 18778 T1260 C116 P56708 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 18778 T1260 C116 P56708 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty3/index.20130131174535088
[junit4:junit4]   2> 18779 T1260 C116 P56708 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty3
[junit4:junit4]   2> 18781 T1260 C116 P56708 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty3/index.20130131174535088 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2d017f57),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 18782 T1260 C116 P56708 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 18782 T1260 C116 P56708 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 18782 T1260 C116 P56708 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty3
[junit4:junit4]   2> 18783 T1260 C116 P56708 oass.SolrIndexSearcher.<init> Opening Searcher@79804ed1 main
[junit4:junit4]   2> 18785 T1259 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@79804ed1 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 18785 T1259 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty3/index
[junit4:junit4]   2> 18785 T1260 C116 P56708 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 18788 T1260 C116 P56708 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty3/index.20130131174535088 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2d017f57),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty3/index.20130131174535088 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2d017f57),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 18788 T1260 C116 P56708 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 18789 T1260 C116 P56708 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty3
[junit4:junit4]   2> 18790 T1260 C116 P56708 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 18790 T1260 C116 P56708 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty3/index.20130131174535088
[junit4:junit4]   2> 18790 T1260 C116 P56708 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty3/index
[junit4:junit4]   2> 18791 T1260 C116 P56708 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 18791 T1260 C116 P56708 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 18791 T1260 C116 P56708 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 18791 T1260 C116 P56708 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 18792 T1186 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 18793 T1260 C116 P56708 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 18795 T1186 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4]   2> 18796 T1186 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19304 T1209 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19306 T1209 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:50178_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:50178"}
[junit4:junit4]   2> 19319 T1209 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:56708_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:56708"}
[junit4:junit4]   2> 19331 T1227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19331 T1214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19331 T1273 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19331 T1242 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19331 T1208 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19331 T1257 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19799 T1186 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 20802 T1186 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C118_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:50178_, base_url=http://127.0.0.1:50178}
[junit4:junit4]   2> 21795 T1277 C118 P50178 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:58857/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 21795 T1277 C118 P50178 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:50178 START replicas=[http://127.0.0.1:58857/collection1/] nUpdates=100
[junit4:junit4]   2> 21796 T1277 C118 P50178 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 21796 T1277 C118 P50178 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 21796 T1277 C118 P50178 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 21796 T1277 C118 P50178 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 21796 T1277 C118 P50178 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 21797 T1277 C118 P50178 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:58857/collection1/. core=collection1
[junit4:junit4]   2> 21797 T1277 C118 P50178 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 21804 T1186 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> ASYNC  NEW_CORE C119 name=collection1 org.apache.solr.core.SolrCore@6931e831 url=http://127.0.0.1:58857/collection1 node=127.0.0.1:58857_ C119_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:58857_, base_url=http://127.0.0.1:58857, leader=true}
[junit4:junit4]   2> 21809 T1233 C119 P58857 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1 
[junit4:junit4]   2> 21810 T1234 C119 P58857 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 21811 T1234 C119 P58857 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty2
[junit4:junit4]   2> 21815 T1234 C119 P58857 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5dd21f15),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 21816 T1234 C119 P58857 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 21820 T1234 C119 P58857 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5dd21f15),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5dd21f15),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 21820 T1234 C119 P58857 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 21821 T1234 C119 P58857 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty2
[junit4:junit4]   2> 21822 T1234 C119 P58857 oass.SolrIndexSearcher.<init> Opening Searcher@13ffa638 realtime
[junit4:junit4]   2> 21822 T1234 C119 P58857 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 21823 T1234 C119 P58857 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 13
[junit4:junit4]   2> 21824 T1277 C118 P50178 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 21824 T1277 C118 P50178 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 21826 T1235 C119 P58857 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 21826 T1235 C119 P58857 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 21827 T1277 C118 P50178 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 21827 T1277 C118 P50178 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 21827 T1277 C118 P50178 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 21829 T1235 C119 P58857 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty2
[junit4:junit4]   2> 21830 T1235 C119 P58857 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty2/index
[junit4:junit4]   2> 21830 T1235 C119 P58857 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1 
[junit4:junit4]   2> 21831 T1277 C118 P50178 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 21832 T1277 C118 P50178 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty4/index.20130131174538155 forceNew:false
[junit4:junit4]   2> 21832 T1277 C118 P50178 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty4
[junit4:junit4]   2> 21833 T1277 C118 P50178 oash.SnapPuller.fetchLatestIndex Starting download to RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty4/index.20130131174538155 lockFactory=org.apache.lucene.store.NativeFSLockFactory@622a702c) fullCopy=true
[junit4:junit4]   2> 21836 T1235 C119 P58857 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> 21838 T1277 C118 P50178 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty4/index
[junit4:junit4]   2> 21838 T1277 C118 P50178 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 21839 T1277 C118 P50178 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130131174538155
[junit4:junit4]   2> 21840 T1277 C118 P50178 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty4
[junit4:junit4]   2> 21841 T1277 C118 P50178 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty4
[junit4:junit4]   2> 21841 T1277 C118 P50178 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty4/index/ new=./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty4/index.20130131174538155
[junit4:junit4]   2> 21844 T1277 C118 P50178 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty4/index.20130131174538155 lockFactory=org.apache.lucene.store.NativeFSLockFactory@622a702c),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 21845 T1277 C118 P50178 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 21846 T1277 C118 P50178 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty4
[junit4:junit4]   2> 21847 T1277 C118 P50178 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 21847 T1277 C118 P50178 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 21847 T1277 C118 P50178 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 21848 T1277 C118 P50178 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty4/index.20130131174538155
[junit4:junit4]   2> 21849 T1277 C118 P50178 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty4
[junit4:junit4]   2> 21852 T1277 C118 P50178 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty4/index.20130131174538155 lockFactory=org.apache.lucene.store.NativeFSLockFactory@622a702c),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 21853 T1277 C118 P50178 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 21853 T1277 C118 P50178 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 21853 T1277 C118 P50178 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty4
[junit4:junit4]   2> 21854 T1277 C118 P50178 oass.SolrIndexSearcher.<init> Opening Searcher@c7516eb main
[junit4:junit4]   2> 21856 T1275 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@c7516eb main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 21856 T1275 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty4/index
[junit4:junit4]   2> 21857 T1277 C118 P50178 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 21859 T1277 C118 P50178 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty4/index.20130131174538155 lockFactory=org.apache.lucene.store.NativeFSLockFactory@622a702c),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty4/index.20130131174538155 lockFactory=org.apache.lucene.store.NativeFSLockFactory@622a702c),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 21860 T1277 C118 P50178 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 21861 T1277 C118 P50178 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty4
[junit4:junit4]   2> 21861 T1277 C118 P50178 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 21861 T1277 C118 P50178 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty4/index.20130131174538155
[junit4:junit4]   2> 21862 T1277 C118 P50178 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty4/index
[junit4:junit4]   2> 21862 T1277 C118 P50178 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 21862 T1277 C118 P50178 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 21862 T1277 C118 P50178 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 21863 T1277 C118 P50178 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 21869 T1277 C118 P50178 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 22342 T1209 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 22344 T1209 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"2",
[junit4:junit4]   2> 	  "shard":"shard2",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:50178_",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:50178"}
[junit4:junit4]   2> 22357 T1214 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 22357 T1208 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 22358 T1242 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 22357 T1227 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 22357 T1273 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 22357 T1257 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 22807 T1186 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 22809 T1186 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4]   2> ASYNC  NEW_CORE C120 name=collection1 org.apache.solr.core.SolrCore@3cd85618 url=http://127.0.0.1:27671/collection1 node=127.0.0.1:27671_ C120_STATE=coll:control_collection core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:27671_, base_url=http://127.0.0.1:27671, leader=true}
[junit4:junit4]   2> 22829 T1198 C120 P27671 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/control/data
[junit4:junit4]   2> 22836 T1198 C120 P27671 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@36a95c7c),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 22837 T1198 C120 P27671 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 22839 T1198 C120 P27671 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/control/data
[junit4:junit4]   2> 22840 T1198 C120 P27671 oass.SolrIndexSearcher.<init> Opening Searcher@ac4626e realtime
[junit4:junit4]   2> 22841 T1198 C120 P27671 /update {wt=javabin&version=2} {deleteByQuery=*:* (-1425700908330647552)} 0 14
[junit4:junit4]   2> 22855 T1221 C117 P52746 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty1
[junit4:junit4]   2> 22880 T1236 C119 P58857 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty2
[junit4:junit4]   2>  C116_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:56708_, base_url=http://127.0.0.1:56708}
[junit4:junit4]   2> 22889 T1247 C116 P56708 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty3
[junit4:junit4]   2> 22891 T1247 C116 P56708 /update {update.distrib=FROMLEADER&_version_=-1425700908357910528&update.from=http://127.0.0.1:52746/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1425700908357910528)} 0 5
[junit4:junit4]   2>  C118_STATE=coll:collection1 core:collection1 props:{shard=shard2, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:50178_, base_url=http://127.0.0.1:50178}
[junit4:junit4]   2> 22896 T1263 C118 P50178 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZkTest-1359654316330/jetty4
[junit4:junit4]   2> 22897 T1263 C118 P50178 /update {update.distrib=FROMLEADER&_version_=-1425700908383076352&update.from=http://127.0.0.1:58857/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1425700908383076352)} 0 4
[junit4:junit4]   2> 22898 T1236 C119 P58857 /update {update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1425700908383076352)} 0 20
[junit4:junit4]   2> 22899 T1221 C117 P52746 /update {wt=javabin&version=2} {deleteByQuery=*:* (-1425700908357910528)} 0 55
[junit4:junit4]   2> 22910 T1198 C120 P27671 /update {wt=javabin&version=2} {add=[1 (1425700908409290752)]} 0 8
[junit4:junit4]   2> 22941 T1247 C116 P56708 /update {distrib.from=http://127.0.0.1:52746/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1425700908436553728)]} 0 5
[junit4:junit4]   2> 22942 T1222 C117 P52746 /update {distrib.from=http://127.0.0.1:58857/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1425700908436553728)]} 0 13
[junit4:junit4]   2> 22943 T1237 C119 P58857 /update {wt=javabin&version=2} {add=[1]} 0 31
[junit4:junit4]   2> 22950 T1198 C120 P27671 /update {wt=javabin&version=2} {add=[2 (1425700908455428096)]} 0 4
[junit4:junit4]   2> 22965 T1263 C118 P50178 /update {distrib.from=http://127.0.0.1:58857/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1425700908461719552)]} 0 5
[junit4:junit4]   2> 22967 T1237 C119 P58857 /update {wt=javabin&version=2} {add=[2 (1425700908461719552)]} 0 14
[junit4:junit4]   2> 22974 T1198 C120 P27671 /update {wt=javabin&version=2} {add=[3 (1425700908481642496)]} 0 3
[junit4:junit4]   2> 22984 T1263 C118 P50178 /update {distrib.from=http://127.0.0.1:58857/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[3 (1425700908486885376)]} 0 2
[junit4:junit4]   2> 22985 T1237 C119 P58857 /update {wt=javabin&version=2} {add=[3 (1425700908486885376)]} 0 8
[junit4:junit4]   2> 22990 T1198 C120 P27671 /update {wt=javabin&version=2} {add=[4 (1425700908498419712)]} 0 2
[junit4:junit4]   2> 23003 T1247 C116 P56708 /update {distrib.from=http://127.0.0.1:52746/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[4 (1425700908506808320)]} 0 2
[junit4:junit4]   2> 23004 T1222 C117 P52746 /update {distrib.from=http://127.0.0.1:58857/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[4 (1425700908506808320)]} 0 8
[junit4:junit4]   2> 23005 T1237 C119 P58857 /update {wt=javabin&version=2} {add=[4]} 0 12
[junit4:junit4]   2> 23010 T1198 C120 P27671 /update {wt=javabin&version=2} {add=[5 (1425700908519391232)]} 0 2
[junit4:junit4]   2> 23023 T1263 C118 P50178 /update {distrib.from=http://127.0.0.1:58857/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[5 (1425700908527779840)]} 0 2
[junit4:junit4]   2> 23024 T1236 C119 P58857 /update {distrib.from=http://127.0.0.1:52746/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[5 (1425700908527779840)]} 0 8
[junit4:junit4]   2> 23025 T1221 C117 P52746 /update {wt=javabin&version=2} {add=[5]} 0 12
[junit4:junit4]   2> 23030 T1198 C120 P27671 /update {wt=javabin&version=2} {add=[6 (1425700908540362752)]} 0 2
[junit4:junit4]   2> 23043 T1263 C118 P50178 /update {distrib.from=http://127.0.0.1:58857/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[6 (1425700908548751360)]} 0 2
[junit4:junit4]   2> 23044 T1236 C119 P58857 /update {distrib.from=http://127.0.0.1:52746/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[6 (1425700908548751360)]} 0 8
[junit4:junit4]   2> 23045 T1221 C117 P52746 /update {wt=javabin&version=2} {add=[6]} 0 12
[junit4:junit4]   2> 23050 T1198 C120 P27671 /update {wt=javabin&version=2} {add=[7 (1425700908561334272)]} 0 2
[junit4:junit4]   2> 23063 T1263 C118 P50178 /update {distrib.from=http://127.0.0.1:58857/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[7 (1425700908568674304)]} 0 3
[junit4:junit4]   2> 23064 T1236 C119 P58857 /update {distrib.from=http://127.0.0.1:52746/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[7 (1425700908568674304)]} 0 9
[junit4:junit4]   2> 23064 T1221 C117 P52746 /update {wt=javabin&version=2} {add=[7]} 0 11
[junit4:junit4]   2> 23070 T1198 C120 P27671 /update {wt=javabin&version=2} {add=[8 (1425700908582305792)]} 0 2
[junit4:junit4]   2> 23082 T1247 C116 P56708 /update {distrib.from=http://127.0.0.1:52746/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[8 (1425700908587548672)]} 0 4
[junit4:junit4]   2> 23084 T1221 C117 P52746 /update {wt=javabin&version=2} {add=[8 (1425700908587548672)]} 0 11
[junit4:junit4]   2> 23090 T1198 C120 P27671 /update {wt=javabin&version=2} {add=[9 (1425700908602228736)]} 0 3
[junit4:junit4]   2> 23102 T1263 C118 P50178 /update {distrib.from=http://127.0.0.1:58857/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[9 (1425700908611665920)]} 0 2
[junit4:junit4]   2> 23103 T1236 C119 P58857 /update {distrib.from=http://127.0.0.1:52746/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[9 (1425700908611665920)]} 0 8
[junit4:junit4]   2> 23105 T1221 C117 P52746 /update {wt=javabin&version=2} {add=[9]} 0 13
[junit4:junit4]   2> 23134 T1198 C120 P27671 /update {wt=javabin&version=2} {add=[10 (1425700908624248832)]} 0 26
[junit4:junit4]   2> 23145 T1247 C116 P56708 /update {distrib.from=http://127.0.0.1:52746/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10 (1425700908655706112)]} 0 3
[junit4:junit4]   2> 23147 T1221 C117 P52746 /update {wt=javabin&version=2} {add=[10 (1425700908655706112)]} 0 10
[junit4:junit4]   2> 23154 T1198 C120 P27671 /update {wt=javabin&version=2} {add=[11 (1425700908668289024)]} 0 4
[junit4:junit4]   2> 23168 T1247 C116 P56708 /update {distrib.from=http://127.0.0.1:52746/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[11 (1425700908678774784)]} 0 3
[junit4:junit4]   2> 23170 T1222 C117 P52746 /update {distrib.from=http://127.0.0.1:58857/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[11 (1425700908678774784)]} 0 10
[junit4:junit4]   2> 23171 T1237 C119 P58857

[...truncated too long message...]

tributedSearchTestCase.java:280)
[junit4:junit4]   2> 	        at org.apache.solr.cloud.AbstractDistribZkTestBase.tearDown(AbstractDistribZkTestBase.java:219)
[junit4:junit4]   2> 	        at org.apache.solr.cloud.AbstractFullDistribZkTestBase.tearDown(AbstractFullDistribZkTestBase.java:1310)
[junit4:junit4]   2> 	        at org.apache.solr.cloud.BasicDistributedZkTest.tearDown(BasicDistributedZkTest.java:1352)
[junit4:junit4]   2> 	        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4:junit4]   2> 	        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[junit4:junit4]   2> 	        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4:junit4]   2> 	        at java.lang.reflect.Method.invoke(Method.java:601)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:795)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4:junit4]   2> 	        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]   2> 	        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]   2> 	        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]   2> 	        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]   2> 	        at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> NOTE: test params are: codec=Lucene41: {n_f1=PostingsFormat(name=Asserting), foo_b=PostingsFormat(name=MockSep), foo_d=PostingsFormat(name=Lucene41WithOrds), foo_f=PostingsFormat(name=MockSep), n_tl1=PostingsFormat(name=Asserting), n_d1=PostingsFormat(name=Memory doPackFST= false), rnd_b=PostingsFormat(name=Lucene41WithOrds), intDefault=PostingsFormat(name=Memory doPackFST= false), n_td1=PostingsFormat(name=Asserting), timestamp=PostingsFormat(name=Memory doPackFST= false), id=PostingsFormat(name=Lucene41WithOrds), range_facet_sl=PostingsFormat(name=MockSep), range_facet_si=PostingsFormat(name=Asserting), oddField_s=PostingsFormat(name=MockSep), foo_i=PostingsFormat(name=Asserting), multiDefault=PostingsFormat(name=Asserting), n_tf1=PostingsFormat(name=Memory doPackFST= false), n_dt1=PostingsFormat(name=Asserting), n_ti1=PostingsFormat(name=Lucene41WithOrds), range_facet_l=PostingsFormat(name=Memory doPackFST= false), text=PostingsFormat(name=MockSep), _version_=PostingsFormat(name=Memory doPackFST= false), val_i=PostingsFormat(name=Lucene41WithOrds), SubjectTerms_mfacet=PostingsFormat(name=Memory doPackFST= false), a_t=PostingsFormat(name=Memory doPackFST= false), n_tdt1=PostingsFormat(name=Memory doPackFST= false), other_tl1=PostingsFormat(name=Memory doPackFST= false), n_l1=PostingsFormat(name=Memory doPackFST= false), a_si=PostingsFormat(name=Asserting)}, sim=DefaultSimilarity, locale=mt_MT, timezone=Atlantic/Faeroe
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=26,free=106423048,total=440401920
[junit4:junit4]   2> NOTE: All tests run in this JVM: [TestCodecSupport, TestLRUCache, TestReplicationHandler, PathHierarchyTokenizerFactoryTest, TestDocumentBuilder, UUIDFieldTest, ShowFileRequestHandlerTest, TestCharFilters, SuggesterTest, TestMultiCoreConfBootstrap, TestPropInject, SynonymTokenizerTest, TestSolrCoreProperties, TestSurroundQueryParser, TestPluginEnable, QueryParsingTest, TestFunctionQuery, TestDistributedGrouping, StatsComponentTest, ZkCLITest, PeerSyncTest, FileUtilsTest, TestBinaryField, DirectSolrSpellCheckerTest, RequiredFieldsTest, TestRecovery, TestDocSet, ConvertedLegacyTest, StatelessScriptUpdateProcessorFactoryTest, SchemaVersionSpecificBehaviorTest, CoreContainerCoreInitFailuresTest, TestJmxIntegration, ZkSolrClientTest, SpellingQueryConverterTest, TestCSVLoader, BasicZkTest, TestMergePolicyConfig, TestReload, DebugComponentTest, TestSolrXMLSerializer, TestReversedWildcardFilterFactory, TestStressLucene, TestRemoteStreaming, LeaderElectionTest, IndexSchemaRuntimeFieldTest, ExternalFileFieldSortTest, TestSolr4Spatial, TestQuerySenderListener, TestRTGBase, SignatureUpdateProcessorFactoryTest, DocumentAnalysisRequestHandlerTest, TestArbitraryIndexDir, TestSolrDeletionPolicy2, TestSolrJ, StandardRequestHandlerTest, BasicFunctionalityTest, TestSolrDeletionPolicy1, TestPhraseSuggestions, UpdateParamsTest, TestLazyCores, ReturnFieldsTest, BasicDistributedZkTest]
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=BasicDistributedZkTest -Dtests.seed=6831F93F68701A73 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=mt_MT -Dtests.timezone=Atlantic/Faeroe -Dtests.file.encoding=US-ASCII
[junit4:junit4] ERROR   0.00s J1 | BasicDistributedZkTest (suite) <<<
[junit4:junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([6831F93F68701A73]:0)
[junit4:junit4] Completed on J1 in 7223.68s, 1 test, 2 errors <<< FAILURES!

[...truncated 11 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:346: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build.xml:178: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/common-build.xml:428: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:1176: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:841: There were test failures: 248 suites, 1036 tests, 1 suite-level error, 1 error, 23 ignored (5 assumptions)

Total time: 145 minutes 51 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure