You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2013/02/20 18:00:31 UTC

[JENKINS] Lucene-Solr-Tests-trunk-java7 - Build # 3749 - Failure

Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-java7/3749/

1 tests failed.
REGRESSION:  org.apache.solr.cloud.SyncSliceTest.testDistribSearch

Error Message:
Test Setup Failure: shard1 should have just been set up to be inconsistent - but it's still consistent. Leader:http://127.0.0.1:56308/zxr/du/collection1skip list:[org.apache.solr.cloud.AbstractFullDistribZkTestBase$CloudJettyRunner@3fbf2092, org.apache.solr.cloud.AbstractFullDistribZkTestBase$CloudJettyRunner@edf423d7]

Stack Trace:
java.lang.AssertionError: Test Setup Failure: shard1 should have just been set up to be inconsistent - but it's still consistent. Leader:http://127.0.0.1:56308/zxr/du/collection1skip list:[org.apache.solr.cloud.AbstractFullDistribZkTestBase$CloudJettyRunner@3fbf2092, org.apache.solr.cloud.AbstractFullDistribZkTestBase$CloudJettyRunner@edf423d7]
	at __randomizedtesting.SeedInfo.seed([E9F4A64CE0DD557F:6812285497823543]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNotNull(Assert.java:526)
	at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:216)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:794)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:722)




Build Log:
[...truncated 8990 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.SyncSliceTest
[junit4:junit4]   2> 0 T649 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /zxr/du
[junit4:junit4]   2> 6 T649 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./solrtest-SyncSliceTest-1361378804112
[junit4:junit4]   2> 7 T649 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4]   2> 7 T650 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4]   2> 11 T650 oazs.ZooKeeperServer.setTickTime tickTime set to 1000
[junit4:junit4]   2> 11 T650 oazs.ZooKeeperServer.setMinSessionTimeout minSessionTimeout set to -1
[junit4:junit4]   2> 11 T650 oazs.ZooKeeperServer.setMaxSessionTimeout maxSessionTimeout set to -1
[junit4:junit4]   2> 11 T650 oazs.NIOServerCnxnFactory.configure binding to port 0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> 12 T650 oazsp.FileTxnSnapLog.save Snapshotting: 0x0 to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/zookeeper/server1/data/version-2/snapshot.0
[junit4:junit4]   2> 108 T649 oasc.ZkTestServer.run start zk server on port:44652
[junit4:junit4]   2> 108 T649 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:44652 sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@6d1e3d1c
[junit4:junit4]   2> 109 T649 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 110 T655 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:44652. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 110 T655 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:44652, initiating session
[junit4:junit4]   2> 111 T651 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:50246
[junit4:junit4]   2> 111 T651 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:50246
[junit4:junit4]   2> 112 T653 oazsp.FileTxnLog.append Creating new log file: log.1
[junit4:junit4]   2> 140 T653 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cf88001980000 with negotiated timeout 10000 for client /140.211.11.196:50246
[junit4:junit4]   2> 140 T655 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:44652, sessionid = 0x13cf88001980000, negotiated timeout = 10000
[junit4:junit4]   2> 140 T656 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6d1e3d1c name:ZooKeeperConnection Watcher:127.0.0.1:44652 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 141 T649 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 141 T649 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4]   2> 145 T654 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13cf88001980000
[junit4:junit4]   2> 153 T656 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 153 T649 oaz.ZooKeeper.close Session: 0x13cf88001980000 closed
[junit4:junit4]   2> 154 T651 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:50246 which had sessionid 0x13cf88001980000
[junit4:junit4]   2> 154 T649 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:44652/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@616b093f
[junit4:junit4]   2> 155 T649 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 155 T657 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:44652. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 156 T657 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:44652, initiating session
[junit4:junit4]   2> 156 T651 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:64773
[junit4:junit4]   2> 156 T651 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:64773
[junit4:junit4]   2> 158 T653 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cf88001980001 with negotiated timeout 10000 for client /140.211.11.196:64773
[junit4:junit4]   2> 158 T657 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:44652, sessionid = 0x13cf88001980001, negotiated timeout = 10000
[junit4:junit4]   2> 158 T658 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@616b093f name:ZooKeeperConnection Watcher:127.0.0.1:44652/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 158 T649 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 159 T649 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4]   2> 169 T649 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4]   2> 173 T649 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4]   2> 176 T649 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4]   2> 179 T649 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 180 T649 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4]   2> 193 T649 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4]   2> 194 T649 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4]   2> 298 T649 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4]   2> 299 T649 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4]   2> 303 T649 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4]   2> 304 T649 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4]   2> 312 T649 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4]   2> 313 T649 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4]   2> 317 T649 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 317 T649 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4]   2> 321 T649 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 321 T649 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4]   2> 325 T649 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 325 T649 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4]   2> 329 T649 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4]   2> 329 T649 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4]   2> 333 T654 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13cf88001980001
[junit4:junit4]   2> 339 T658 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 340 T651 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:64773 which had sessionid 0x13cf88001980001
[junit4:junit4]   2> 339 T649 oaz.ZooKeeper.close Session: 0x13cf88001980001 closed
[junit4:junit4]   2> 512 T649 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 515 T649 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:45461
[junit4:junit4]   2> 517 T649 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 517 T649 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 518 T649 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1361378804446
[junit4:junit4]   2> 518 T649 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1361378804446/solr.xml
[junit4:junit4]   2> 519 T649 oasc.CoreContainer.<init> New CoreContainer 1955364523
[junit4:junit4]   2> 519 T649 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1361378804446/'
[junit4:junit4]   2> 520 T649 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1361378804446/'
[junit4:junit4]   2> 581 T649 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 582 T649 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 582 T649 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 583 T649 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 583 T649 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 583 T649 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 584 T649 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 584 T649 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 585 T649 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 585 T649 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 609 T649 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 629 T649 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:44652/solr
[junit4:junit4]   2> 629 T649 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 630 T649 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:44652 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@70dca9a7
[junit4:junit4]   2> 631 T649 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 631 T668 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:44652. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 632 T668 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:44652, initiating session
[junit4:junit4]   2> 632 T651 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:39441
[junit4:junit4]   2> 633 T651 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:39441
[junit4:junit4]   2> 635 T653 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cf88001980002 with negotiated timeout 20000 for client /140.211.11.196:39441
[junit4:junit4]   2> 635 T668 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:44652, sessionid = 0x13cf88001980002, negotiated timeout = 20000
[junit4:junit4]   2> 635 T669 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@70dca9a7 name:ZooKeeperConnection Watcher:127.0.0.1:44652 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 635 T649 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 637 T654 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13cf88001980002
[junit4:junit4]   2> 647 T669 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 647 T651 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:39441 which had sessionid 0x13cf88001980002
[junit4:junit4]   2> 647 T649 oaz.ZooKeeper.close Session: 0x13cf88001980002 closed
[junit4:junit4]   2> 647 T649 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 661 T649 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:44652/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@1296144
[junit4:junit4]   2> 662 T649 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 662 T670 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:44652. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 662 T670 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:44652, initiating session
[junit4:junit4]   2> 663 T651 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:17689
[junit4:junit4]   2> 663 T651 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:17689
[junit4:junit4]   2> 664 T653 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cf88001980003 with negotiated timeout 20000 for client /140.211.11.196:17689
[junit4:junit4]   2> 664 T670 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:44652, sessionid = 0x13cf88001980003, negotiated timeout = 20000
[junit4:junit4]   2> 665 T671 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1296144 name:ZooKeeperConnection Watcher:127.0.0.1:44652/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 665 T649 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 672 T654 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cf88001980003 type:create cxid:0x2 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 674 T649 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4]   2> 677 T649 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45461_zxr%2Fdu
[junit4:junit4]   2> 677 T654 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cf88001980003 type:delete cxid:0x8 zxid:0x1c txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:45461_zxr%2Fdu Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:45461_zxr%2Fdu
[junit4:junit4]   2> 683 T649 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:45461_zxr%2Fdu
[junit4:junit4]   2> 687 T649 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4]   2> 700 T654 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cf88001980003 type:delete cxid:0x16 zxid:0x21 txntype:-1 reqpath:n/a Error Path:/solr/overseer_elect/leader Error:KeeperErrorCode = NoNode for /solr/overseer_elect/leader
[junit4:junit4]   2> 701 T649 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4]   2> 705 T649 oasc.Overseer.start Overseer (id=89219321306808323-127.0.0.1:45461_zxr%2Fdu-n_0000000000) starting
[junit4:junit4]   2> 705 T654 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cf88001980003 type:create cxid:0x1b zxid:0x23 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 712 T654 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cf88001980003 type:create cxid:0x1c zxid:0x24 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 714 T654 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cf88001980003 type:create cxid:0x1d zxid:0x25 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 720 T654 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cf88001980003 type:create cxid:0x1e zxid:0x26 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 721 T673 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4]   2> 722 T649 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4]   2> 732 T649 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 734 T672 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4]   2> 739 T674 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1361378804446/collection1
[junit4:junit4]   2> 740 T674 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4]   2> 740 T674 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 740 T674 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4]   2> 742 T674 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1361378804446/collection1/'
[junit4:junit4]   2> 743 T674 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1361378804446/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 744 T674 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1361378804446/collection1/lib/README' to classloader
[junit4:junit4]   2> 794 T674 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 855 T674 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 956 T674 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 973 T674 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 1619 T674 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 1642 T674 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 1647 T674 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 1665 T674 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1670 T674 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 1675 T674 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1676 T674 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 1678 T674 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1361378804446/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1361378804112/control/data/
[junit4:junit4]   2> 1678 T674 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@70e3b9b4
[junit4:junit4]   2> 1679 T674 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 1714 T674 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/control/data forceNew:false
[junit4:junit4]   2> 1715 T674 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/control/data
[junit4:junit4]   2> 1715 T674 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1361378804112/control/data/index/
[junit4:junit4]   2> 1716 T674 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1361378804112/control/data/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 1717 T674 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/control/data/index forceNew:false
[junit4:junit4]   2> 1723 T674 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1361378804112/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5f80cd83; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 1724 T674 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 1725 T674 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/control/data/index
[junit4:junit4]   2> 1726 T674 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 1727 T674 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 1727 T674 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 1728 T674 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 1728 T674 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 1729 T674 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 1729 T674 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 1729 T674 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 1730 T674 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 1742 T674 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 1748 T674 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/control/data
[junit4:junit4]   2> 1749 T674 oass.SolrIndexSearcher.<init> Opening Searcher@68fe0c41 main
[junit4:junit4]   2> 1750 T674 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 1751 T674 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 1754 T675 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@68fe0c41 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 1756 T674 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 1756 T674 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 1756 T654 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cf88001980003 type:create cxid:0x50 zxid:0x29 txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue Error:KeeperErrorCode = NoNode for /solr/overseer/queue
[junit4:junit4]   2> 2238 T672 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 2238 T672 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:45461_zxr%2Fdu",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:45461/zxr/du"}
[junit4:junit4]   2> 2238 T672 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4]   2> 2239 T672 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 2239 T654 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cf88001980003 type:create cxid:0x58 zxid:0x2c txntype:-1 reqpath:n/a Error Path:/solr/overseer/queue-work Error:KeeperErrorCode = NoNode for /solr/overseer/queue-work
[junit4:junit4]   2> 2257 T671 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 2771 T674 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 2771 T674 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:45461/zxr/du collection:control_collection shard:shard1
[junit4:junit4]   2> 2773 T674 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4]   2> 2791 T654 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cf88001980003 type:delete cxid:0x72 zxid:0x36 txntype:-1 reqpath:n/a Error Path:/solr/collections/control_collection/leaders Error:KeeperErrorCode = NoNode for /solr/collections/control_collection/leaders
[junit4:junit4]   2> 2792 T674 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 2793 T654 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cf88001980003 type:create cxid:0x73 zxid:0x37 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 2804 T674 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 2804 T674 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 2804 T674 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:45461/zxr/du/collection1/
[junit4:junit4]   2> 2804 T674 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 2804 T674 oasc.SyncStrategy.syncToMe http://127.0.0.1:45461/zxr/du/collection1/ has no replicas
[junit4:junit4]   2> 2805 T674 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:45461/zxr/du/collection1/
[junit4:junit4]   2> 2805 T674 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4]   2> 2810 T654 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cf88001980003 type:create cxid:0x7d zxid:0x3b txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 3771 T672 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3810 T671 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4]   2> 3840 T674 oasc.ZkController.register We are http://127.0.0.1:45461/zxr/du/collection1/ and leader is http://127.0.0.1:45461/zxr/du/collection1/
[junit4:junit4]   2> 3840 T674 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:45461/zxr/du
[junit4:junit4]   2> 3840 T674 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 3840 T674 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 3841 T674 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 3842 T674 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 3843 T649 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 3844 T649 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 3845 T649 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 3860 T649 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4]   2> 3861 T649 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:44652/solr sessionTimeout=10000 watcher=org.apache.solr.common.cloud.ConnectionManager@7ca6ff7d
[junit4:junit4]   2> 3862 T649 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 3863 T676 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:44652. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 3863 T676 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:44652, initiating session
[junit4:junit4]   2> 3864 T651 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:47979
[junit4:junit4]   2> 3864 T651 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:47979
[junit4:junit4]   2> 3865 T653 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cf88001980004 with negotiated timeout 10000 for client /140.211.11.196:47979
[junit4:junit4]   2> 3865 T676 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:44652, sessionid = 0x13cf88001980004, negotiated timeout = 10000
[junit4:junit4]   2> 3866 T677 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7ca6ff7d name:ZooKeeperConnection Watcher:127.0.0.1:44652/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 3866 T649 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 3868 T649 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 3869 T649 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4]   2> 4047 T649 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 4050 T649 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:54537
[junit4:junit4]   2> 4050 T649 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 4051 T649 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 4051 T649 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1361378807976
[junit4:junit4]   2> 4052 T649 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1361378807976/solr.xml
[junit4:junit4]   2> 4052 T649 oasc.CoreContainer.<init> New CoreContainer 1177712449
[junit4:junit4]   2> 4053 T649 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1361378807976/'
[junit4:junit4]   2> 4054 T649 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1361378807976/'
[junit4:junit4]   2> 4116 T649 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 4116 T649 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 4117 T649 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 4117 T649 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 4117 T649 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 4118 T649 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 4118 T649 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 4119 T649 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 4119 T649 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 4120 T649 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 4141 T649 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 4162 T649 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:44652/solr
[junit4:junit4]   2> 4162 T649 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 4163 T649 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:44652 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@5e27c35a
[junit4:junit4]   2> 4164 T649 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4165 T687 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:44652. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4165 T687 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:44652, initiating session
[junit4:junit4]   2> 4166 T651 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:33761
[junit4:junit4]   2> 4166 T651 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:33761
[junit4:junit4]   2> 4168 T653 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cf88001980005 with negotiated timeout 20000 for client /140.211.11.196:33761
[junit4:junit4]   2> 4168 T687 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:44652, sessionid = 0x13cf88001980005, negotiated timeout = 20000
[junit4:junit4]   2> 4169 T688 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5e27c35a name:ZooKeeperConnection Watcher:127.0.0.1:44652 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4169 T649 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4170 T654 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13cf88001980005
[junit4:junit4]   2> 4184 T688 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 4184 T651 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:33761 which had sessionid 0x13cf88001980005
[junit4:junit4]   2> 4184 T649 oaz.ZooKeeper.close Session: 0x13cf88001980005 closed
[junit4:junit4]   2> 4185 T649 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 4196 T649 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:44652/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@56f1488f
[junit4:junit4]   2> 4197 T649 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 4197 T689 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:44652. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 4198 T689 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:44652, initiating session
[junit4:junit4]   2> 4198 T651 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:27962
[junit4:junit4]   2> 4198 T651 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:27962
[junit4:junit4]   2> 4200 T653 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cf88001980006 with negotiated timeout 20000 for client /140.211.11.196:27962
[junit4:junit4]   2> 4200 T689 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:44652, sessionid = 0x13cf88001980006, negotiated timeout = 20000
[junit4:junit4]   2> 4200 T690 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@56f1488f name:ZooKeeperConnection Watcher:127.0.0.1:44652/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 4200 T649 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 4201 T654 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cf88001980006 type:create cxid:0x1 zxid:0x49 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4203 T654 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cf88001980006 type:create cxid:0x2 zxid:0x4a txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 4211 T649 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 5214 T649 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54537_zxr%2Fdu
[junit4:junit4]   2> 5215 T654 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cf88001980006 type:delete cxid:0x9 zxid:0x4b txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:54537_zxr%2Fdu Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:54537_zxr%2Fdu
[junit4:junit4]   2> 5217 T649 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:54537_zxr%2Fdu
[junit4:junit4]   2> 5220 T671 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 5221 T677 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 5221 T690 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4]   2> 5221 T671 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5239 T691 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1361378807976/collection1
[junit4:junit4]   2> 5239 T691 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 5240 T691 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 5240 T691 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 5242 T691 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1361378807976/collection1/'
[junit4:junit4]   2> 5243 T691 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1361378807976/collection1/lib/README' to classloader
[junit4:junit4]   2> 5244 T691 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1361378807976/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 5296 T691 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 5325 T672 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 5326 T672 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"control_collection",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:45461_zxr%2Fdu",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:45461/zxr/du"}
[junit4:junit4]   2> 5339 T671 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5339 T690 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5339 T677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 5361 T691 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 5462 T691 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 5479 T691 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 6131 T691 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 6143 T691 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 6147 T691 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 6165 T691 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 6170 T691 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 6175 T691 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 6177 T691 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 6178 T691 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1361378807976/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty1/
[junit4:junit4]   2> 6178 T691 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@70e3b9b4
[junit4:junit4]   2> 6179 T691 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 6180 T691 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty1 forceNew:false
[junit4:junit4]   2> 6180 T691 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty1
[junit4:junit4]   2> 6181 T691 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty1/index/
[junit4:junit4]   2> 6181 T691 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 6182 T691 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty1/index forceNew:false
[junit4:junit4]   2> 6189 T691 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3725d2a; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 6189 T691 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 6190 T691 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty1/index
[junit4:junit4]   2> 6193 T691 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 6193 T691 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 6194 T691 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 6194 T691 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 6195 T691 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 6195 T691 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 6196 T691 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 6196 T691 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 6197 T691 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 6210 T691 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 6216 T691 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty1
[junit4:junit4]   2> 6218 T691 oass.SolrIndexSearcher.<init> Opening Searcher@68f11765 main
[junit4:junit4]   2> 6219 T691 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 6219 T691 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 6224 T692 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@68f11765 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 6226 T691 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 6226 T691 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 6845 T672 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 6846 T672 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:54537_zxr%2Fdu",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:54537/zxr/du"}
[junit4:junit4]   2> 6846 T672 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 1
[junit4:junit4]   2> 6846 T672 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 6852 T671 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6852 T677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 6852 T690 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 7229 T691 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 7229 T691 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:54537/zxr/du collection:collection1 shard:shard1
[junit4:junit4]   2> 7230 T691 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4]   2> 7247 T654 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cf88001980006 type:delete cxid:0x46 zxid:0x5b txntype:-1 reqpath:n/a Error Path:/solr/collections/collection1/leaders Error:KeeperErrorCode = NoNode for /solr/collections/collection1/leaders
[junit4:junit4]   2> 7249 T691 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4]   2> 7249 T654 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cf88001980006 type:create cxid:0x47 zxid:0x5c txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 7257 T691 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4]   2> 7258 T691 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4]   2> 7258 T691 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:54537/zxr/du/collection1/
[junit4:junit4]   2> 7258 T691 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4]   2> 7258 T691 oasc.SyncStrategy.syncToMe http://127.0.0.1:54537/zxr/du/collection1/ has no replicas
[junit4:junit4]   2> 7259 T691 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:54537/zxr/du/collection1/
[junit4:junit4]   2> 7259 T691 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4]   2> 7263 T654 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cf88001980006 type:create cxid:0x51 zxid:0x60 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8368 T672 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8453 T671 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8453 T677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8453 T690 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 8497 T691 oasc.ZkController.register We are http://127.0.0.1:54537/zxr/du/collection1/ and leader is http://127.0.0.1:54537/zxr/du/collection1/
[junit4:junit4]   2> 8497 T691 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:54537/zxr/du
[junit4:junit4]   2> 8497 T691 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4]   2> 8497 T691 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 8497 T691 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 8499 T691 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 8500 T649 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 8501 T649 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 8502 T649 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 8693 T649 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 8696 T649 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:56308
[junit4:junit4]   2> 8697 T649 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 8697 T649 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 8698 T649 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1361378812623
[junit4:junit4]   2> 8698 T649 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1361378812623/solr.xml
[junit4:junit4]   2> 8699 T649 oasc.CoreContainer.<init> New CoreContainer 678903041
[junit4:junit4]   2> 8700 T649 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1361378812623/'
[junit4:junit4]   2> 8700 T649 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1361378812623/'
[junit4:junit4]   2> 8761 T649 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 8762 T649 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 8762 T649 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 8763 T649 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 8763 T649 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 8764 T649 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 8764 T649 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 8764 T649 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 8765 T649 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 8765 T649 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 8788 T649 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 8809 T649 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:44652/solr
[junit4:junit4]   2> 8809 T649 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 8810 T649 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:44652 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@3b40d63
[junit4:junit4]   2> 8811 T649 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 8811 T702 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:44652. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 8812 T702 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:44652, initiating session
[junit4:junit4]   2> 8813 T651 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:41716
[junit4:junit4]   2> 8813 T651 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:41716
[junit4:junit4]   2> 8840 T653 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cf88001980007 with negotiated timeout 20000 for client /140.211.11.196:41716
[junit4:junit4]   2> 8840 T702 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:44652, sessionid = 0x13cf88001980007, negotiated timeout = 20000
[junit4:junit4]   2> 8840 T703 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3b40d63 name:ZooKeeperConnection Watcher:127.0.0.1:44652 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 8841 T649 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 8842 T654 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13cf88001980007
[junit4:junit4]   2> 8874 T703 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 8874 T651 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:41716 which had sessionid 0x13cf88001980007
[junit4:junit4]   2> 8874 T649 oaz.ZooKeeper.close Session: 0x13cf88001980007 closed
[junit4:junit4]   2> 8875 T649 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 8888 T649 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:44652/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@4253076f
[junit4:junit4]   2> 8889 T649 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 8890 T704 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:44652. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 8890 T704 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:44652, initiating session
[junit4:junit4]   2> 8890 T651 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:23066
[junit4:junit4]   2> 8891 T651 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:23066
[junit4:junit4]   2> 8895 T653 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cf88001980008 with negotiated timeout 20000 for client /140.211.11.196:23066
[junit4:junit4]   2> 8895 T704 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:44652, sessionid = 0x13cf88001980008, negotiated timeout = 20000
[junit4:junit4]   2> 8895 T705 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4253076f name:ZooKeeperConnection Watcher:127.0.0.1:44652/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 8895 T649 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 8896 T654 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cf88001980008 type:create cxid:0x1 zxid:0x6d txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8898 T654 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cf88001980008 type:create cxid:0x2 zxid:0x6e txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 8908 T649 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 9911 T649 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56308_zxr%2Fdu
[junit4:junit4]   2> 9912 T654 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cf88001980008 type:delete cxid:0x9 zxid:0x6f txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:56308_zxr%2Fdu Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:56308_zxr%2Fdu
[junit4:junit4]   2> 9914 T649 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:56308_zxr%2Fdu
[junit4:junit4]   2> 9917 T690 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4]   2> 9918 T671 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 9918 T705 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 9918 T677 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 9918 T671 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9918 T677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9920 T690 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4]   2> 9937 T706 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1361378812623/collection1
[junit4:junit4]   2> 9937 T706 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 9938 T706 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 9938 T706 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 9940 T706 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1361378812623/collection1/'
[junit4:junit4]   2> 9941 T706 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1361378812623/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 9941 T706 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1361378812623/collection1/lib/README' to classloader
[junit4:junit4]   2> 9983 T672 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 9984 T672 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:54537_zxr%2Fdu",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:54537/zxr/du"}
[junit4:junit4]   2> 9998 T671 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9998 T705 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9998 T677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 9998 T690 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 10006 T706 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 10067 T706 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 10168 T706 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 10185 T706 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 10795 T706 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 10803 T706 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 10806 T706 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 10820 T706 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 10825 T706 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 10829 T706 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 10830 T706 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 10831 T706 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1361378812623/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty2/
[junit4:junit4]   2> 10831 T706 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@70e3b9b4
[junit4:junit4]   2> 10832 T706 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 10832 T706 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty2 forceNew:false
[junit4:junit4]   2> 10833 T706 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty2
[junit4:junit4]   2> 10833 T706 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty2/index/
[junit4:junit4]   2> 10833 T706 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 10834 T706 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty2/index forceNew:false
[junit4:junit4]   2> 10839 T706 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2751fa28; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 10839 T706 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 10840 T706 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty2/index
[junit4:junit4]   2> 10842 T706 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 10842 T706 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 10842 T706 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 10843 T706 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 10843 T706 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 10843 T706 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 10844 T706 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 10844 T706 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 10844 T706 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 10853 T706 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 10857 T706 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty2
[junit4:junit4]   2> 10858 T706 oass.SolrIndexSearcher.<init> Opening Searcher@54a5de37 main
[junit4:junit4]   2> 10859 T706 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 10859 T706 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 10862 T707 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@54a5de37 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 10864 T706 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 10864 T706 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 11504 T672 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11505 T672 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:56308_zxr%2Fdu",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:56308/zxr/du"}
[junit4:junit4]   2> 11505 T672 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4]   2> 11505 T672 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 11510 T705 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11510 T671 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11510 T677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11510 T690 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 11867 T706 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 11867 T706 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:56308/zxr/du collection:collection1 shard:shard1
[junit4:junit4]   2> 11870 T706 oasc.ZkController.register We are http://127.0.0.1:56308/zxr/du/collection1/ and leader is http://127.0.0.1:54537/zxr/du/collection1/
[junit4:junit4]   2> 11870 T706 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:56308/zxr/du
[junit4:junit4]   2> 11871 T706 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 11871 T706 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C32 name=collection1 org.apache.solr.core.SolrCore@74688f0e url=http://127.0.0.1:56308/zxr/du/collection1 node=127.0.0.1:56308_zxr%2Fdu C32_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:56308_zxr%2Fdu, base_url=http://127.0.0.1:56308/zxr/du}
[junit4:junit4]   2> 11872 T708 C32 P56308 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 11872 T706 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 11873 T708 C32 P56308 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 11873 T708 C32 P56308 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 11873 T708 C32 P56308 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 11873 T649 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 11874 T649 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 11875 T708 C32 P56308 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 11875 T649 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 12071 T649 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 12074 T649 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:44737
[junit4:junit4]   2> 12075 T649 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 12075 T649 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 12075 T649 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1361378816002
[junit4:junit4]   2> 12076 T649 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1361378816002/solr.xml
[junit4:junit4]   2> 12076 T649 oasc.CoreContainer.<init> New CoreContainer 2091780920
[junit4:junit4]   2> 12077 T649 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1361378816002/'
[junit4:junit4]   2> 12078 T649 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1361378816002/'
[junit4:junit4]   2> 12137 T649 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 12138 T649 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 12138 T649 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 12139 T649 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 12139 T649 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 12140 T649 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 12140 T649 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 12141 T649 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 12141 T649 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 12142 T649 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 12163 T649 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 12182 T649 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:44652/solr
[junit4:junit4]   2> 12183 T649 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 12184 T649 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:44652 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@47dacbf
[junit4:junit4]   2> 12185 T649 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 12185 T718 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:44652. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 12186 T718 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:44652, initiating session
[junit4:junit4]   2> 12186 T651 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:62169
[junit4:junit4]   2> 12186 T651 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:62169
[junit4:junit4]   2> 12188 T653 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cf88001980009 with negotiated timeout 20000 for client /140.211.11.196:62169
[junit4:junit4]   2> 12188 T718 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:44652, sessionid = 0x13cf88001980009, negotiated timeout = 20000
[junit4:junit4]   2> 12189 T719 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@47dacbf name:ZooKeeperConnection Watcher:127.0.0.1:44652 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 12189 T649 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 12191 T654 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13cf88001980009
[junit4:junit4]   2> 12192 T719 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 12192 T651 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:62169 which had sessionid 0x13cf88001980009
[junit4:junit4]   2> 12192 T649 oaz.ZooKeeper.close Session: 0x13cf88001980009 closed
[junit4:junit4]   2> 12193 T649 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 12204 T649 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:44652/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@45946c34
[junit4:junit4]   2> 12205 T649 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 12205 T720 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:44652. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 12205 T720 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:44652, initiating session
[junit4:junit4]   2> 12206 T651 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:17433
[junit4:junit4]   2> 12206 T651 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:17433
[junit4:junit4]   2> 12207 T653 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cf8800198000a with negotiated timeout 20000 for client /140.211.11.196:17433
[junit4:junit4]   2> 12207 T720 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:44652, sessionid = 0x13cf8800198000a, negotiated timeout = 20000
[junit4:junit4]   2> 12208 T721 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@45946c34 name:ZooKeeperConnection Watcher:127.0.0.1:44652/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 12208 T649 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 12209 T654 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cf8800198000a type:create cxid:0x1 zxid:0x80 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12210 T654 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cf8800198000a type:create cxid:0x2 zxid:0x81 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 12212 T649 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2> 13026 T672 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 13027 T672 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:56308_zxr%2Fdu",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:56308/zxr/du"}
[junit4:junit4]   2> 13046 T721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13046 T677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13046 T690 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13046 T705 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13046 T671 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13215 T649 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44737_zxr%2Fdu
[junit4:junit4]   2> 13216 T654 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cf8800198000a type:delete cxid:0xb zxid:0x86 txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:44737_zxr%2Fdu Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:44737_zxr%2Fdu
[junit4:junit4]   2> 13217 T649 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:44737_zxr%2Fdu
[junit4:junit4]   2> 13220 T690 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4]   2> 13220 T721 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 13220 T677 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 13220 T721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13220 T671 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 13220 T705 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 13221 T671 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13221 T677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13221 T705 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 13222 T690 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4]   2> 13237 T722 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1361378816002/collection1
[junit4:junit4]   2> 13237 T722 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 13238 T722 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 13238 T722 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 13239 T722 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1361378816002/collection1/'
[junit4:junit4]   2> 13240 T722 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1361378816002/collection1/lib/README' to classloader
[junit4:junit4]   2> 13241 T722 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1361378816002/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 13295 T722 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 13368 T722 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 13468 T722 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 13484 T722 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 14137 T722 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 14149 T722 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 14153 T722 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 14171 T722 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 14176 T722 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 14181 T722 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 14183 T722 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 14184 T722 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1361378816002/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty3/
[junit4:junit4]   2> 14184 T722 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@70e3b9b4
[junit4:junit4]   2> 14185 T722 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 14186 T722 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty3 forceNew:false
[junit4:junit4]   2> 14186 T722 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty3
[junit4:junit4]   2> 14187 T722 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty3/index/
[junit4:junit4]   2> 14187 T722 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 14188 T722 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty3/index forceNew:false
[junit4:junit4]   2> 14195 T722 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7d492d7a; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 14195 T722 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 14196 T722 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty3/index
[junit4:junit4]   2> 14199 T722 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 14199 T722 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 14200 T722 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 14200 T722 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 14201 T722 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 14201 T722 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 14202 T722 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 14202 T722 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 14203 T722 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 14214 T722 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 14218 T722 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty3
[junit4:junit4]   2> 14220 T722 oass.SolrIndexSearcher.<init> Opening Searcher@7c2006e6 main
[junit4:junit4]   2> 14220 T722 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 14221 T722 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 14224 T723 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7c2006e6 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 14226 T722 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 14226 T722 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 14552 T672 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 14552 T672 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:44737_zxr%2Fdu",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:44737/zxr/du"}
[junit4:junit4]   2> 14553 T672 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4]   2> 14553 T672 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 14557 T721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14558 T677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14558 T705 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14557 T671 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 14557 T690 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 15229 T722 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 15229 T722 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:44737/zxr/du collection:collection1 shard:shard1
[junit4:junit4]   2> 15233 T722 oasc.ZkController.register We are http://127.0.0.1:44737/zxr/du/collection1/ and leader is http://127.0.0.1:54537/zxr/du/collection1/
[junit4:junit4]   2> 15233 T722 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:44737/zxr/du
[junit4:junit4]   2> 15233 T722 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 15233 T722 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C33 name=collection1 org.apache.solr.core.SolrCore@32970686 url=http://127.0.0.1:44737/zxr/du/collection1 node=127.0.0.1:44737_zxr%2Fdu C33_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:44737_zxr%2Fdu, base_url=http://127.0.0.1:44737/zxr/du}
[junit4:junit4]   2> 15234 T724 C33 P44737 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 15234 T722 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 15235 T724 C33 P44737 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 15235 T724 C33 P44737 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 15235 T724 C33 P44737 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 15236 T649 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 15236 T649 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 15237 T724 C33 P44737 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 15237 T649 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 15425 T649 oejs.Server.doStart jetty-8.1.8.v20121106
[junit4:junit4]   2> 15429 T649 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:47834
[junit4:junit4]   2> 15430 T649 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4]   2> 15430 T649 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4]   2> 15431 T649 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1361378819360
[junit4:junit4]   2> 15431 T649 oasc.CoreContainer$Initializer.initialize looking for solr.xml: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1361378819360/solr.xml
[junit4:junit4]   2> 15431 T649 oasc.CoreContainer.<init> New CoreContainer 157813887
[junit4:junit4]   2> 15432 T649 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1361378819360/'
[junit4:junit4]   2> 15433 T649 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1361378819360/'
[junit4:junit4]   2> 15493 T649 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4]   2> 15493 T649 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4]   2> 15494 T649 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4]   2> 15494 T649 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4]   2> 15495 T649 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4]   2> 15495 T649 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4]   2> 15495 T649 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4]   2> 15496 T649 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4]   2> 15496 T649 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4]   2> 15497 T649 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4]   2> 15520 T649 oasc.CoreContainer.load Registering Log Listener
[junit4:junit4]   2> 15540 T649 oasc.CoreContainer.initZooKeeper Zookeeper client=127.0.0.1:44652/solr
[junit4:junit4]   2> 15540 T649 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4]   2> 15541 T649 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:44652 sessionTimeout=60000 watcher=org.apache.solr.common.cloud.ConnectionManager@582b9d0f
[junit4:junit4]   2> 15542 T649 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 15542 T734 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:44652. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 15543 T734 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:44652, initiating session
[junit4:junit4]   2> 15543 T651 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:46026
[junit4:junit4]   2> 15544 T651 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:46026
[junit4:junit4]   2> 15546 T653 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cf8800198000b with negotiated timeout 20000 for client /140.211.11.196:46026
[junit4:junit4]   2> 15546 T734 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:44652, sessionid = 0x13cf8800198000b, negotiated timeout = 20000
[junit4:junit4]   2> 15546 T735 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@582b9d0f name:ZooKeeperConnection Watcher:127.0.0.1:44652 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 15546 T649 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 15548 T654 oazs.PrepRequestProcessor.pRequest2Txn Processed session termination for sessionid: 0x13cf8800198000b
[junit4:junit4]   2> 15562 T735 oaz.ClientCnxn$EventThread.run EventThread shut down
[junit4:junit4]   2> 15562 T651 oazs.NIOServerCnxn.closeSock Closed socket connection for client /140.211.11.196:46026 which had sessionid 0x13cf8800198000b
[junit4:junit4]   2> 15562 T649 oaz.ZooKeeper.close Session: 0x13cf8800198000b closed
[junit4:junit4]   2> 15562 T649 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4]   2> 15575 T649 oaz.ZooKeeper.<init> Initiating client connection, connectString=127.0.0.1:44652/solr sessionTimeout=30000 watcher=org.apache.solr.common.cloud.ConnectionManager@338df84c
[junit4:junit4]   2> 15576 T649 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4]   2> 15577 T736 oaz.ClientCnxn$SendThread.logStartConnect Opening socket connection to server localhost/127.0.0.1:44652. Will not attempt to authenticate using SASL (access denied ("javax.security.auth.AuthPermission" "getLoginConfiguration"))
[junit4:junit4]   2> 15577 T736 oaz.ClientCnxn$SendThread.primeConnection Socket connection established to localhost/127.0.0.1:44652, initiating session
[junit4:junit4]   2> 15578 T651 oazs.NIOServerCnxnFactory.run Accepted socket connection from /140.211.11.196:38545
[junit4:junit4]   2> 15578 T651 oazs.ZooKeeperServer.processConnectRequest Client attempting to establish new session at /140.211.11.196:38545
[junit4:junit4]   2> 15579 T653 oazs.ZooKeeperServer.finishSessionInit Established session 0x13cf8800198000c with negotiated timeout 20000 for client /140.211.11.196:38545
[junit4:junit4]   2> 15579 T736 oaz.ClientCnxn$SendThread.onConnected Session establishment complete on server localhost/127.0.0.1:44652, sessionid = 0x13cf8800198000c, negotiated timeout = 20000
[junit4:junit4]   2> 15580 T737 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@338df84c name:ZooKeeperConnection Watcher:127.0.0.1:44652/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4]   2> 15580 T649 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4]   2> 15581 T654 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cf8800198000c type:create cxid:0x1 zxid:0x93 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 15583 T654 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cf8800198000c type:create cxid:0x2 zxid:0x94 txntype:-1 reqpath:n/a Error Path:/solr/overseer Error:KeeperErrorCode = NodeExists for /solr/overseer
[junit4:junit4]   2> 15590 T649 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
[junit4:junit4]   2>  C32_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:56308_zxr%2Fdu, base_url=http://127.0.0.1:56308/zxr/du}
[junit4:junit4]   2> 15901 T708 C32 P56308 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:54537/zxr/du/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 15901 T708 C32 P56308 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:56308/zxr/du START replicas=[http://127.0.0.1:54537/zxr/du/collection1/] nUpdates=100
[junit4:junit4]   2> 15902 T708 C32 P56308 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 15902 T708 C32 P56308 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 15902 T708 C32 P56308 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 15903 T708 C32 P56308 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 15903 T708 C32 P56308 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 15903 T708 C32 P56308 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:54537/zxr/du/collection1/. core=collection1
[junit4:junit4]   2> 15903 T708 C32 P56308 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> ASYNC  NEW_CORE C34 name=collection1 org.apache.solr.core.SolrCore@7fdec3b url=http://127.0.0.1:54537/zxr/du/collection1 node=127.0.0.1:54537_zxr%2Fdu C34_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:54537_zxr%2Fdu, base_url=http://127.0.0.1:54537/zxr/du, leader=true}
[junit4:junit4]   2> 15909 T682 C34 P54537 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 15918 T683 C34 P54537 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 15919 T683 C34 P54537 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty1
[junit4:junit4]   2> 15923 T683 C34 P54537 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3725d2a; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 15924 T683 C34 P54537 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 15928 T683 C34 P54537 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3725d2a; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3725d2a; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 15929 T683 C34 P54537 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 15930 T683 C34 P54537 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty1
[junit4:junit4]   2> 15931 T683 C34 P54537 oass.SolrIndexSearcher.<init> Opening Searcher@18222222 realtime
[junit4:junit4]   2> 15931 T683 C34 P54537 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15931 T683 C34 P54537 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 14
[junit4:junit4]   2> 15932 T708 C32 P56308 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 15932 T708 C32 P56308 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 15935 T684 C34 P54537 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 15935 T684 C34 P54537 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15936 T708 C32 P56308 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4]   2> 15936 T708 C32 P56308 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 15936 T708 C32 P56308 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 15938 T684 C34 P54537 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty1
[junit4:junit4]   2> 15939 T684 C34 P54537 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty1/index
[junit4:junit4]   2> 15939 T684 C34 P54537 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1 
[junit4:junit4]   2> 15940 T708 C32 P56308 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 15941 T708 C32 P56308 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty2/index.20130220094700046 forceNew:false
[junit4:junit4]   2> 15942 T708 C32 P56308 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty2
[junit4:junit4]   2> 15942 T708 C32 P56308 oash.SnapPuller.fetchLatestIndex Starting download to NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty2/index.20130220094700046 lockFactory=org.apache.lucene.store.NativeFSLockFactory@d237481; maxCacheMB=48.0 maxMergeSizeMB=4.0) fullCopy=true
[junit4:junit4]   2> 15946 T684 C34 P54537 REQ /replication {file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0 
[junit4:junit4]   2> 15948 T708 C32 P56308 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty2/index
[junit4:junit4]   2> 15948 T708 C32 P56308 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 15948 T708 C32 P56308 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130220094700046
[junit4:junit4]   2> 15950 T708 C32 P56308 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty2
[junit4:junit4]   2> 15950 T708 C32 P56308 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty2
[junit4:junit4]   2> 15950 T708 C32 P56308 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty2/index/ new=./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty2/index.20130220094700046
[junit4:junit4]   2> 15954 T708 C32 P56308 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty2/index.20130220094700046 lockFactory=org.apache.lucene.store.NativeFSLockFactory@d237481; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 15954 T708 C32 P56308 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 15956 T708 C32 P56308 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty2
[junit4:junit4]   2> 15956 T708 C32 P56308 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 15956 T708 C32 P56308 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 15956 T708 C32 P56308 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 15957 T708 C32 P56308 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty2/index.20130220094700046
[junit4:junit4]   2> 15958 T708 C32 P56308 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty2
[junit4:junit4]   2> 15961 T708 C32 P56308 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty2/index.20130220094700046 lockFactory=org.apache.lucene.store.NativeFSLockFactory@d237481; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 15962 T708 C32 P56308 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 15962 T708 C32 P56308 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 15962 T708 C32 P56308 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty2
[junit4:junit4]   2> 15963 T708 C32 P56308 oass.SolrIndexSearcher.<init> Opening Searcher@1bfa50cf main
[junit4:junit4]   2> 15964 T707 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1bfa50cf main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 15964 T707 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty2/index
[junit4:junit4]   2> 15965 T708 C32 P56308 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 15968 T708 C32 P56308 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty2/index.20130220094700046 lockFactory=org.apache.lucene.store.NativeFSLockFactory@d237481; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty2/index.20130220094700046 lockFactory=org.apache.lucene.store.NativeFSLockFactory@d237481; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 15968 T708 C32 P56308 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 15969 T708 C32 P56308 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty2
[junit4:junit4]   2> 15970 T708 C32 P56308 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 15970 T708 C32 P56308 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty2/index.20130220094700046
[junit4:junit4]   2> 15970 T708 C32 P56308 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty2/index
[junit4:junit4]   2> 15970 T708 C32 P56308 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 15970 T708 C32 P56308 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 15971 T708 C32 P56308 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 15971 T708 C32 P56308 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 15978 T708 C32 P56308 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 16073 T672 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 16074 T672 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:44737_zxr%2Fdu",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:44737/zxr/du"}
[junit4:junit4]   2> 16078 T672 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:56308_zxr%2Fdu",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:56308/zxr/du"}
[junit4:junit4]   2> 16090 T705 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16090 T671 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16090 T690 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16090 T677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16090 T737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16090 T721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16593 T649 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47834_zxr%2Fdu
[junit4:junit4]   2> 16594 T654 oazs.PrepRequestProcessor.pRequest Got user-level KeeperException when processing sessionid:0x13cf8800198000c type:delete cxid:0xb zxid:0x9d txntype:-1 reqpath:n/a Error Path:/solr/live_nodes/127.0.0.1:47834_zxr%2Fdu Error:KeeperErrorCode = NoNode for /solr/live_nodes/127.0.0.1:47834_zxr%2Fdu
[junit4:junit4]   2> 16595 T649 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:47834_zxr%2Fdu
[junit4:junit4]   2> 16598 T737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16598 T690 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4]   2> 16598 T721 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 16598 T705 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 16599 T677 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 16599 T705 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16599 T671 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 16599 T721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16599 T677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16599 T671 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 16600 T737 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 16610 T690 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4]   2> 16620 T740 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1361378819360/collection1
[junit4:junit4]   2> 16620 T740 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4]   2> 16620 T740 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4]   2> 16621 T740 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4]   2> 16638 T740 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1361378819360/collection1/'
[junit4:junit4]   2> 16640 T740 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty4-1361378819360/collection1/lib/classes/' to classloader
[junit4:junit4]   2> 16640 T740 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty4-1361378819360/collection1/lib/README' to classloader
[junit4:junit4]   2> 16714 T740 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4]   2> 16774 T740 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4]   2> 16874 T740 oass.IndexSchema.readSchema Reading Solr Schema
[junit4:junit4]   2> 16887 T740 oass.IndexSchema.readSchema Schema name=test
[junit4:junit4]   2> 17540 T740 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4]   2> 17552 T740 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4]   2> 17556 T740 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4]   2> 17574 T740 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 17580 T740 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4]   2> 17585 T740 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 17586 T740 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4]   2> 17587 T740 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1361378819360/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty4/
[junit4:junit4]   2> 17588 T740 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@70e3b9b4
[junit4:junit4]   2> 17588 T740 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4]   2> 17589 T740 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty4 forceNew:false
[junit4:junit4]   2> 17590 T740 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty4
[junit4:junit4]   2> 17590 T740 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty4/index/
[junit4:junit4]   2> 17591 T740 oasc.SolrCore.initIndex WARNING [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4]   2> 17592 T740 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty4/index forceNew:false
[junit4:junit4]   2> 17624 T740 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1bfee644; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 17625 T740 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 17626 T740 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty4/index
[junit4:junit4]   2> 17628 T740 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4]   2> 17629 T740 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4]   2> 17629 T740 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4]   2> 17630 T740 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4]   2> 17630 T740 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4]   2> 17631 T740 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4]   2> 17631 T740 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4]   2> 17632 T740 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4]   2> 17632 T740 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4]   2> 17645 T740 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4]   2> 17651 T740 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty4
[junit4:junit4]   2> 17653 T740 oass.SolrIndexSearcher.<init> Opening Searcher@2914ff70 main
[junit4:junit4]   2> 17654 T740 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4]   2> 17654 T740 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4]   2> 17659 T741 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2914ff70 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 17661 T740 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4]   2> 17661 T740 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2>  C33_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:44737_zxr%2Fdu, base_url=http://127.0.0.1:44737/zxr/du}
[junit4:junit4]   2> 18258 T724 C33 P44737 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:54537/zxr/du/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 18258 T724 C33 P44737 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:44737/zxr/du START replicas=[http://127.0.0.1:54537/zxr/du/collection1/] nUpdates=100
[junit4:junit4]   2> 18259 T724 C33 P44737 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 18259 T724 C33 P44737 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 18259 T724 C33 P44737 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 18259 T724 C33 P44737 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 18259 T724 C33 P44737 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 18260 T724 C33 P44737 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:54537/zxr/du/collection1/. core=collection1
[junit4:junit4]   2> 18260 T724 C33 P44737 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 18261 T682 C34 P54537 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0 
[junit4:junit4]   2> 18277 T685 C34 P54537 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 18302 T685 C34 P54537 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3725d2a; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3725d2a; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 18303 T685 C34 P54537 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 18303 T685 C34 P54537 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty1
[junit4:junit4]   2> 18304 T685 C34 P54537 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 18304 T685 C34 P54537 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 28
[junit4:junit4]   2> 18305 T724 C33 P44737 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 18305 T724 C33 P44737 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 18308 T684 C34 P54537 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 18309 T724 C33 P44737 oash.SnapPuller.fetchLatestIndex Master's generation: 3
[junit4:junit4]   2> 18309 T724 C33 P44737 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 18309 T724 C33 P44737 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 18311 T684 C34 P54537 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty1
[junit4:junit4]   2> 18312 T684 C34 P54537 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty1/index
[junit4:junit4]   2> 18312 T684 C34 P54537 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=3&version=2} status=0 QTime=1 
[junit4:junit4]   2> 18313 T724 C33 P44737 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 18315 T724 C33 P44737 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty3/index.20130220094702420 forceNew:false
[junit4:junit4]   2> 18316 T724 C33 P44737 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty3
[junit4:junit4]   2> 18317 T724 C33 P44737 oash.SnapPuller.fetchLatestIndex Starting download to NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty3/index.20130220094702420 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5464cba3; maxCacheMB=48.0 maxMergeSizeMB=4.0) fullCopy=true
[junit4:junit4]   2> 18322 T684 C34 P54537 REQ /replication {file=segments_3&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=3} status=0 QTime=0 
[junit4:junit4]   2> 18325 T724 C33 P44737 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty3/index
[junit4:junit4]   2> 18325 T724 C33 P44737 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 18326 T724 C33 P44737 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130220094702420
[junit4:junit4]   2> 18327 T724 C33 P44737 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty3
[junit4:junit4]   2> 18328 T724 C33 P44737 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty3
[junit4:junit4]   2> 18328 T724 C33 P44737 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty3/index/ new=./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty3/index.20130220094702420
[junit4:junit4]   2> 18331 T724 C33 P44737 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty3/index.20130220094702420 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5464cba3; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 18332 T724 C33 P44737 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 18333 T724 C33 P44737 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty3
[junit4:junit4]   2> 18333 T724 C33 P44737 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 18334 T724 C33 P44737 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 18334 T724 C33 P44737 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 18335 T724 C33 P44737 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty3/index.20130220094702420
[junit4:junit4]   2> 18335 T724 C33 P44737 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty3
[junit4:junit4]   2> 18339 T724 C33 P44737 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty3/index.20130220094702420 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5464cba3; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 18339 T724 C33 P44737 oasc.SolrDeletionPolicy.updateCommits newest commit = 3[segments_3]
[junit4:junit4]   2> 18339 T724 C33 P44737 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 18340 T724 C33 P44737 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty3
[junit4:junit4]   2> 18340 T724 C33 P44737 oass.SolrIndexSearcher.<init> Opening Searcher@5bea4188 main
[junit4:junit4]   2> 18341 T723 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5bea4188 main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4]   2> 18342 T723 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty3/index
[junit4:junit4]   2> 18342 T724 C33 P44737 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 18363 T724 C33 P44737 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty3/index.20130220094702420 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5464cba3; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty3/index.20130220094702420 lockFactory=org.apache.lucene.store.NativeFSLockFactory@5464cba3; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4,filenames=[segments_4]
[junit4:junit4]   2> 18363 T724 C33 P44737 oasc.SolrDeletionPolicy.updateCommits newest commit = 4[segments_4]
[junit4:junit4]   2> 18365 T724 C33 P44737 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty3
[junit4:junit4]   2> 18365 T724 C33 P44737 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 18365 T724 C33 P44737 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty3/index.20130220094702420
[junit4:junit4]   2> 18365 T724 C33 P44737 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty3/index
[junit4:junit4]   2> 18365 T724 C33 P44737 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 18366 T724 C33 P44737 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 18366 T724 C33 P44737 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 18366 T724 C33 P44737 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 18429 T724 C33 P44737 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 19125 T672 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19126 T672 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":null,
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"down",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:47834_zxr%2Fdu",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:47834/zxr/du"}
[junit4:junit4]   2> 19126 T672 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4]   2> 19127 T672 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4]   2> 19131 T672 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:44737_zxr%2Fdu",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:44737/zxr/du"}
[junit4:junit4]   2> 19149 T705 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19149 T677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19149 T671 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19149 T737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19149 T721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19149 T690 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 19665 T740 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4]   2> 19665 T740 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:47834/zxr/du collection:collection1 shard:shard1
[junit4:junit4]   2> 19669 T740 oasc.ZkController.register We are http://127.0.0.1:47834/zxr/du/collection1/ and leader is http://127.0.0.1:54537/zxr/du/collection1/
[junit4:junit4]   2> 19669 T740 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:47834/zxr/du
[junit4:junit4]   2> 19669 T740 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4]   2> 19669 T740 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4]   2> ASYNC  NEW_CORE C35 name=collection1 org.apache.solr.core.SolrCore@2785f44f url=http://127.0.0.1:47834/zxr/du/collection1 node=127.0.0.1:47834_zxr%2Fdu C35_STATE=coll:collection1 core:collection1 props:{shard=null, roles=null, state=down, core=collection1, collection=collection1, node_name=127.0.0.1:47834_zxr%2Fdu, base_url=http://127.0.0.1:47834/zxr/du}
[junit4:junit4]   2> 19670 T744 C35 P47834 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
[junit4:junit4]   2> 19671 T744 C35 P47834 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4]   2> 19671 T740 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19671 T744 C35 P47834 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4]   2> 19671 T744 C35 P47834 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 19672 T649 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0
[junit4:junit4]   2> 19673 T649 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4]   2> 19673 T649 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 19675 T744 C35 P47834 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 19691 T649 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 19710 T649 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4]   2> 19711 T649 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4]   2> 19711 T649 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 20662 T672 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 20663 T672 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"recovering",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:47834_zxr%2Fdu",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:47834/zxr/du"}
[junit4:junit4]   2> 20668 T721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20668 T737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20668 T677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20668 T671 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20668 T705 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20668 T690 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 20713 T649 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 21715 T649 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2>  C35_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:47834_zxr%2Fdu, base_url=http://127.0.0.1:47834/zxr/du}
[junit4:junit4]   2> 22694 T744 C35 P47834 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:54537/zxr/du/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4]   2> 22694 T744 C35 P47834 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:47834/zxr/du START replicas=[http://127.0.0.1:54537/zxr/du/collection1/] nUpdates=100
[junit4:junit4]   2> 22718 T649 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 22819 T744 C35 P47834 oasu.PeerSync.sync WARNING no frame of reference to tell of we've missed updates
[junit4:junit4]   2> 22819 T744 C35 P47834 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4]   2> 22819 T744 C35 P47834 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4]   2> 22820 T744 C35 P47834 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4]   2> 22820 T744 C35 P47834 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4]   2> 22820 T744 C35 P47834 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:54537/zxr/du/collection1/. core=collection1
[junit4:junit4]   2> 22820 T744 C35 P47834 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4]   2> 22823 T682 C34 P54537 REQ /get {getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1 
[junit4:junit4]   2> 22835 T680 C34 P54537 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 22912 T680 C34 P54537 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=3
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3725d2a; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3725d2a; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3725d2a; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4,filenames=[segments_4]
[junit4:junit4]   2> 22913 T680 C34 P54537 oasc.SolrDeletionPolicy.updateCommits newest commit = 4[segments_4]
[junit4:junit4]   2> 22914 T680 C34 P54537 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty1
[junit4:junit4]   2> 22915 T680 C34 P54537 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 22915 T680 C34 P54537 /update {waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 80
[junit4:junit4]   2> 22916 T744 C35 P47834 oash.ReplicationHandler.inform Commits will be reserved for  10000
[junit4:junit4]   2> 22917 T744 C35 P47834 oash.SnapPuller.<init>  No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4]   2> 22919 T684 C34 P54537 REQ /replication {command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0 
[junit4:junit4]   2> 22920 T744 C35 P47834 oash.SnapPuller.fetchLatestIndex Master's generation: 4
[junit4:junit4]   2> 22920 T744 C35 P47834 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4]   2> 22921 T744 C35 P47834 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4]   2> 22923 T684 C34 P54537 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty1
[junit4:junit4]   2> 22924 T684 C34 P54537 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty1/index
[junit4:junit4]   2> 22924 T684 C34 P54537 REQ /replication {command=filelist&qt=/replication&wt=javabin&generation=4&version=2} status=0 QTime=1 
[junit4:junit4]   2> 22925 T744 C35 P47834 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4]   2> 22927 T744 C35 P47834 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty4/index.20130220094707032 forceNew:false
[junit4:junit4]   2> 22927 T744 C35 P47834 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty4
[junit4:junit4]   2> 22928 T744 C35 P47834 oash.SnapPuller.fetchLatestIndex Starting download to NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty4/index.20130220094707032 lockFactory=org.apache.lucene.store.NativeFSLockFactory@279d9a84; maxCacheMB=48.0 maxMergeSizeMB=4.0) fullCopy=true
[junit4:junit4]   2> 22932 T684 C34 P54537 REQ /replication {file=segments_4&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=4} status=0 QTime=0 
[junit4:junit4]   2> 22934 T744 C35 P47834 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty4/index
[junit4:junit4]   2> 22934 T744 C35 P47834 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4]   2> 22935 T744 C35 P47834 oash.SnapPuller.modifyIndexProps New index installed. Updating index properties... index=index.20130220094707032
[junit4:junit4]   2> 22937 T744 C35 P47834 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty4
[junit4:junit4]   2> 22937 T744 C35 P47834 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty4
[junit4:junit4]   2> 22937 T744 C35 P47834 oasc.SolrCore.getNewIndexDir New index directory detected: old=./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty4/index/ new=./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty4/index.20130220094707032
[junit4:junit4]   2> 22941 T744 C35 P47834 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty4/index.20130220094707032 lockFactory=org.apache.lucene.store.NativeFSLockFactory@279d9a84; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4,filenames=[segments_4]
[junit4:junit4]   2> 22942 T744 C35 P47834 oasc.SolrDeletionPolicy.updateCommits newest commit = 4[segments_4]
[junit4:junit4]   2> 22943 T744 C35 P47834 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty4
[junit4:junit4]   2> 22944 T744 C35 P47834 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4]   2> 22944 T744 C35 P47834 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4]   2> 22944 T744 C35 P47834 oasu.DefaultSolrCoreState.newIndexWriter Rollback old IndexWriter... core=collection1
[junit4:junit4]   2> 22945 T744 C35 P47834 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty4/index.20130220094707032
[junit4:junit4]   2> 22946 T744 C35 P47834 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty4
[junit4:junit4]   2> 22950 T744 C35 P47834 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty4/index.20130220094707032 lockFactory=org.apache.lucene.store.NativeFSLockFactory@279d9a84; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4,filenames=[segments_4]
[junit4:junit4]   2> 22951 T744 C35 P47834 oasc.SolrDeletionPolicy.updateCommits newest commit = 4[segments_4]
[junit4:junit4]   2> 22951 T744 C35 P47834 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4]   2> 22951 T744 C35 P47834 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty4
[junit4:junit4]   2> 22952 T744 C35 P47834 oass.SolrIndexSearcher.<init> Opening Searcher@7fdbf03e main
[junit4:junit4]   2> 22953 T741 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7fdbf03e main{StandardDirectoryReader(segments_4:1:nrt)}
[junit4:junit4]   2> 22954 T741 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty4/index
[junit4:junit4]   2> 22954 T744 C35 P47834 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=false,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 22958 T744 C35 P47834 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty4/index.20130220094707032 lockFactory=org.apache.lucene.store.NativeFSLockFactory@279d9a84; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4,filenames=[segments_4]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty4/index.20130220094707032 lockFactory=org.apache.lucene.store.NativeFSLockFactory@279d9a84; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5,filenames=[segments_5]
[junit4:junit4]   2> 22958 T744 C35 P47834 oasc.SolrDeletionPolicy.updateCommits newest commit = 5[segments_5]
[junit4:junit4]   2> 22959 T744 C35 P47834 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty4
[junit4:junit4]   2> 22960 T744 C35 P47834 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 22960 T744 C35 P47834 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty4/index.20130220094707032
[junit4:junit4]   2> 22960 T744 C35 P47834 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty4/index
[junit4:junit4]   2> 22960 T744 C35 P47834 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4]   2> 22960 T744 C35 P47834 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4]   2> 22961 T744 C35 P47834 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4]   2> 22961 T744 C35 P47834 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4]   2> 22969 T744 C35 P47834 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4]   2> 23685 T672 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 23686 T672 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4]   2> 	  "operation":"state",
[junit4:junit4]   2> 	  "numShards":"1",
[junit4:junit4]   2> 	  "shard":"shard1",
[junit4:junit4]   2> 	  "roles":null,
[junit4:junit4]   2> 	  "state":"active",
[junit4:junit4]   2> 	  "core":"collection1",
[junit4:junit4]   2> 	  "collection":"collection1",
[junit4:junit4]   2> 	  "node_name":"127.0.0.1:47834_zxr%2Fdu",
[junit4:junit4]   2> 	  "base_url":"http://127.0.0.1:47834/zxr/du"}
[junit4:junit4]   2> 23703 T721 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 23703 T690 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 23703 T737 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 23703 T671 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 23703 T677 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 23703 T705 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4]   2> 23720 T649 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
[junit4:junit4]   2> 23721 T649 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4]   2> ASYNC  NEW_CORE C36 name=collection1 org.apache.solr.core.SolrCore@51f1d753 url=http://127.0.0.1:45461/zxr/du/collection1 node=127.0.0.1:45461_zxr%2Fdu C36_STATE=coll:control_collection core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:45461_zxr%2Fdu, base_url=http://127.0.0.1:45461/zxr/du, leader=true}
[junit4:junit4]   2> 23733 T661 C36 P45461 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 23734 T661 C36 P45461 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/control/data
[junit4:junit4]   2> 23739 T661 C36 P45461 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1361378804112/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5f80cd83; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 23740 T661 C36 P45461 oasc.SolrDeletionPolicy.updateCommits newest commit = 1[segments_1]
[junit4:junit4]   2> 23748 T661 C36 P45461 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1361378804112/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5f80cd83; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1361378804112/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@5f80cd83; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 23749 T661 C36 P45461 oasc.SolrDeletionPolicy.updateCommits newest commit = 2[segments_2]
[junit4:junit4]   2> 23750 T661 C36 P45461 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/control/data
[junit4:junit4]   2> 23752 T661 C36 P45461 oass.SolrIndexSearcher.<init> Opening Searcher@d5ff264 main
[junit4:junit4]   2> 23753 T661 C36 P45461 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 23754 T675 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@d5ff264 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4]   2> 23754 T675 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/control/data/index
[junit4:junit4]   2> 23755 T661 C36 P45461 /update {waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 22
[junit4:junit4]   2> 23767 T683 C34 P54537 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 23772 T683 C34 P54537 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=4
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3725d2a; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3725d2a; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3725d2a; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4,filenames=[segments_4]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@3725d2a; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5,filenames=[segments_5]
[junit4:junit4]   2> 23772 T683 C34 P54537 oasc.SolrDeletionPolicy.updateCommits newest commit = 5[segments_5]
[junit4:junit4]   2> 23773 T683 C34 P54537 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty1
[junit4:junit4]   2> 23774 T683 C34 P54537 oass.SolrIndexSearcher.<init> Opening Searcher@3ff0d92f main
[junit4:junit4]   2> 23774 T683 C34 P54537 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty1/index
[junit4:junit4]   2> 23775 T683 C34 P54537 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4]   2> 23776 T692 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3ff0d92f main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4]   2> 23776 T692 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty1/index
[junit4:junit4]   2> 23777 T683 C34 P54537 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:56308/zxr/du/collection1/, StdNode: http://127.0.0.1:44737/zxr/du/collection1/, StdNode: http://127.0.0.1:47834/zxr/du/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4]   2>  C35_STATE=coll:collection1 core:collection1 props:{shard=shard1, roles=null, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:47834_zxr%2Fdu, base_url=http://127.0.0.1:47834/zxr/du}
[junit4:junit4]   2> 23793 T727 C35 P47834 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false}
[junit4:junit4]   2> 23798 T727 C35 P47834 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty4/index.20130220094707032 lockFactory=org.apache.lucene.store.NativeFSLockFactory@279d9a84; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5,filenames=[segments_5]
[junit4:junit4]   2> 		commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty4/index.20130220094707032 lockFactory=org.apache.lucene.store.NativeFSLockFactory@279d9a84; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_6,generation=6,filenames=[segments_6]
[junit4:junit4]   2> 23799 T727 C35 P47834 oasc.SolrDeletionPolicy.updateCommits newest commit = 6[segments_6]
[junit4:junit4]   2> 23801 T727 C35 P47834 oasc.CachingDirectoryFactory.close Releasing directory:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-1361378804112/jetty4
[junit4:junit4]   2> 23802 T727 C35 P47834 oass.SolrIndexSearcher.<init> O

[...truncated too long message...]

tdown of request processor complete
[junit4:junit4]   2> 133978 T649 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:44652 44652
[junit4:junit4]   2> 134106 T651 oazs.NIOServerCnxnFactory.run NIOServerCnxn factory exited run method
[junit4:junit4]   2> 134107 T649 oazs.ZooKeeperServer.shutdown shutting down
[junit4:junit4]   2> 134107 T649 oazs.SessionTrackerImpl.shutdown Shutting down
[junit4:junit4]   2> 134107 T649 oazs.PrepRequestProcessor.shutdown Shutting down
[junit4:junit4]   2> 134108 T649 oazs.SyncRequestProcessor.shutdown Shutting down
[junit4:junit4]   2> 134108 T649 oazs.FinalRequestProcessor.shutdown shutdown of request processor complete
[junit4:junit4]   1> client port:0.0.0.0/0.0.0.0:0
[junit4:junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=SyncSliceTest -Dtests.method=testDistribSearch -Dtests.seed=E9F4A64CE0DD557F -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ja -Dtests.timezone=America/Creston -Dtests.file.encoding=UTF-8
[junit4:junit4] FAILURE  134s J0 | SyncSliceTest.testDistribSearch <<<
[junit4:junit4]    > Throwable #1: java.lang.AssertionError: Test Setup Failure: shard1 should have just been set up to be inconsistent - but it's still consistent. Leader:http://127.0.0.1:56308/zxr/du/collection1skip list:[org.apache.solr.cloud.AbstractFullDistribZkTestBase$CloudJettyRunner@3fbf2092, org.apache.solr.cloud.AbstractFullDistribZkTestBase$CloudJettyRunner@edf423d7]
[junit4:junit4]    > 	at __randomizedtesting.SeedInfo.seed([E9F4A64CE0DD557F:6812285497823543]:0)
[junit4:junit4]    > 	at org.junit.Assert.fail(Assert.java:93)
[junit4:junit4]    > 	at org.junit.Assert.assertTrue(Assert.java:43)
[junit4:junit4]    > 	at org.junit.Assert.assertNotNull(Assert.java:526)
[junit4:junit4]    > 	at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:216)
[junit4:junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:794)
[junit4:junit4]    > 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4:junit4]    > 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[junit4:junit4]    > 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4:junit4]    > 	at java.lang.reflect.Method.invoke(Method.java:601)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
[junit4:junit4]    > 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
[junit4:junit4]    > 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
[junit4:junit4]    > 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4:junit4]    > 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
[junit4:junit4]    > 	at java.lang.Thread.run(Thread.java:722)
[junit4:junit4]   2> 134256 T649 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4]   2> 134261 T648 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4:junit4]   2> 134895 T652 oazs.SessionTrackerImpl.run SessionTrackerImpl exited loop!
[junit4:junit4]   2> NOTE: test params are: codec=Lucene42: {range_facet_l=PostingsFormat(name=NestedPulsing), text=MockFixedIntBlock(blockSize=228), _version_=PostingsFormat(name=Memory doPackFST= false), rnd_b=PostingsFormat(name=MockSep), intDefault=PostingsFormat(name=Memory doPackFST= false), timestamp=PostingsFormat(name=NestedPulsing), id=PostingsFormat(name=MockSep), a_t=PostingsFormat(name=NestedPulsing), range_facet_sl=PostingsFormat(name=MockSep), range_facet_si=PostingsFormat(name=NestedPulsing), other_tl1=PostingsFormat(name=Memory doPackFST= false), multiDefault=PostingsFormat(name=NestedPulsing), a_si=PostingsFormat(name=Memory doPackFST= false)}, docValues:{timestamp=DocValuesFormat(name=CheapBastard)}, sim=DefaultSimilarity, locale=ja, timezone=America/Creston
[junit4:junit4]   2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation 1.7.0_09 (64-bit)/cpus=16,threads=5,free=80026168,total=164626432
[junit4:junit4]   2> NOTE: All tests run in this JVM: [SolrPluginUtilsTest, SpellCheckCollatorTest, FieldAnalysisRequestHandlerTest, ChaosMonkeySafeLeaderTest, DistanceFunctionTest, AutoCommitTest, FileBasedSpellCheckerTest, TestAtomicUpdateErrorCases, SpellPossibilityIteratorTest, DateMathParserTest, MBeansHandlerTest, SolrCoreCheckLockOnStartupTest, TestRandomFaceting, TestLuceneMatchVersion, JSONWriterTest, TestFastLRUCache, UpdateRequestProcessorFactoryTest, SpellCheckComponentTest, SolrRequestParserTest, NumericFieldsTest, SpatialFilterTest, TestFastOutputStream, WordBreakSolrSpellCheckerTest, UniqFieldsUpdateProcessorFactoryTest, TestXIncludeConfig, SyncSliceTest]
[junit4:junit4] Completed on J0 in 135.04s, 1 test, 1 failure <<< FAILURES!

[...truncated 554 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:381: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:361: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/build.xml:183: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/solr/common-build.xml:447: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:1202: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:865: There were test failures: 254 suites, 1052 tests, 1 failure, 22 ignored (4 assumptions)

Total time: 48 minutes 24 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure